slop-grader
Jev-AI CLI tool that evaluates text against custom rulesets
81 followers
Jev-AI CLI tool that evaluates text against custom rulesets
81 followers
slop-grader is a rule-based CLI tool that evaluates documents against custom rulesets and produces document scores and line-by-line flags to guide auto-fixing with an AI agent. Use it to catch AI filler in launch copy, to strip buzzwords from landing pages, score narrative flow in launch emails. Flagged lines let agents draft sharper copy. The tool flags outputs prompts so agents can draft fixes.
slop-grader
Can the output be used directly in an AI coding or writing agent workflow or is it mainly meant to be reviewed manually?
slop-grader
@ben_d3 The tool outputs a list of flagged lines and instructions that you can paste directly into an AI agent to fix the document.
How do you handle words that might be considered buzzwords in one industry but are completely normal in another?
slop-grader
@lucyjane you can create custom rules for your usecase :-)
slop-grader
@lucyjane The real power comes from creating custom rules for your usecase. Rules are questions evaluated line by line ("Does this line make a promise that requires a legal disclaimer?") or across the entire document ("Does the opening earn the reader's next 30 seconds?"). You can use the built-in skill (https://github.com/lukstei/slop-grader/blob/main/skills/create-slop-grader-rules/SKILL.md) to create custom rules.
It could be useful to have a strictness setting for the ruleset so users can choose between a quick cleanup and a much deeper review.
Dial
the Du vs Sie consistency example is a good one because it shows this isn't just an "AI filler" detector, it's closer to a linter for anything you can phrase as a yes/no question about a line. we ship voice agent scripts and the equivalent problem for us is formality drift across a long call flow - a script written over multiple sessions slowly shifts tone without anyone noticing until a user points it out. one question on the output format: when a rule fires on a line, does slop-grader give you just the line number and the rule text, or does it also explain why it thinks the rule was violated? that's usually the gap between "useful lint" and "lint you end up ignoring"
slop-grader
@galdayan It does not give you the reason why it's fired, but you can think of creating separate rules for each check, the AI agent is then very good at inferring the problem. Every rule is matched again every line separately, but since it's using this special kind of AI model it's very cheap and fast to evaluate every rule separately.
The line by line flags are a useful touch. It makes it much easier to see exactly what needs fixing instead of rewriting the whole document.