#333 Sherlock Phase 3: Implement Semantic Idempotency
Description
EditPHASE 3: Semantic Idempotency (Efficiency)
IMPLEMENTATION: Option 1 - Embedding-based Semantic Similarity
ARCHITECTURE:
1. Create tools.semantic.embed - Generate embeddings via Ollama (nomic-embed-text)
2. Create tools.semantic.compare - Cosine similarity comparison
3. Add semantic_check parameter to ctx.step() - opt-in only
4. Store embeddings in checkpoint state alongside result
5. On replay: compute new embedding, compare similarity, log result
6. MUST run in Activity Workers only (avoid DB connection exhaustion)
STORAGE:
- Checkpoint state: {result, schema_hash, embedding, similarity_threshold}
- Comparison logged: {old_embedding_hash, new_embedding_hash, similarity_score, passed}
WORKFLOW EXAMPLE:
builder.activity(
task_id='summarize_text',
function='tools.llm.call',
kwargs={...},
semantic_check=True,
similarity_threshold=0.95
)
REQUIREMENTS:
- All verifications in logs/checkpoints
- Activity workers only
- Opt-in (semantic_check=True)
- No regression on atomicity tests
- DOCUMENTATION: Update docs/HIGHWAY_WORKFLOW_ENGINE_BOOK.md
- TESTING: Use ollama nomic-embed-text model
Comments
Loading comments...
Context
Loading context...
Audit History
View AllLoading audit history...