• 4 Posts
  • 177 Comments
Joined 1 year ago
cake
Cake day: June 17th, 2023

help-circle




  • This is the kind of AI stuff that really annoys me. Looking at one of the mutation examples I didn’t see anything that wouldn’t normally be tested by a typical mutation tool. You took a simple, idempotent process and you got an llm to do it slower, less accurately, and using more resources.

    If you wanted to marry the two in a new and possibly useful fashion I would say use an llm to analyze the results of a standard mutation test and give guidance on what issues should be acted upon first. An off-by-one calculation could mean somebody loses a million dollars or it could mean a button is grayed out. Standard mutation tools don’t give you that context.














  • Deliberate was gone for a little while (available in HuggingFace now) and I think I’ve seen others be taken down too for violating licenses or the author doing shady things to boost stats. Some older models are also really hard to find. Even though they’re low quality by current standards they did seem to have a bit more creativity.

    Might be misremember on the subscription thing. Maybe it was v1-v2 of some model was free to download and anything newer was API only. Regardless I think authors are free to pull stuff whenever they choose.