Before install
Confirm the publisher identity and repository URL match the listing. Spoofed “official” SEO plugins already show up in noisy directories.
Skim LICENSE and SECURITY.md if they exist. No license plus broad network permissions is a reason to reject it for production machines.
Check last release date against Claude Code’s recent breaking changes. Stale plugins fail in confusing ways.
During install
Record every permission prompt. If the plugin wants unrestricted filesystem write “for convenience,” ask what file scope it truly needs.
Run the documented hello-world command on a throwaway repo first. If docs skip a minimal example, assume support will be painful.
Note cold-start time and token burn on a small task. A skill that uses a large token budget to rewrite a README is inefficient, not powerful.
After first real task
Did outputs include verifiable claims (URLs, file paths, diffs) or unsupported assertions?
Did it respect your stack (Next.js, WordPress, monorepo) or give generic advice?
Would you trust it unsupervised overnight? If not, keep it in supervised use until the scorecard improves.
Ship your own plugin? Reverse the checklist
Authors should pass the same bar: crisp install, honest permissions, current docs, and sample commands that work on day one. Pair this with the listing quality guide so marketplace visitors see what auditors will eventually measure.