Tool to create a new metric in HoneyHive. Use when you need to define how to evaluate model outputs, whether through code (PYTHON), AI evaluation (LLM), human review (HUMAN), or combining multiple metrics (COMPOSITE). Important: LLM metrics require both model_provider and model_…
nameRequiredstringName of the metric. Must be unique within the project and descriptive of what is being measured.
typeRequiredstringType of metric evaluation. PYTHON: code-based, LLM: AI-evaluated, HUMAN: manually evaluated, COMPOSITE: combination of other metrics.
scaleintegerScale for rating-type metrics (e.g., 1-5 scale, 1-10 scale)
filtersobjectFilter conditions to apply before computing the metric
criteriaRequiredstringEvaluation criteria that defines what this metric measures and how it should be assessed. Required for all metric types.
thresholdobjectThreshold settings for determining when a metric passes or fails.
categoriesarrayList of categories with scores for categorical metrics. Required when return_type is 'categorical'.
model_namestringSpecific model name for LLM-evaluated metrics (e.g., 'gpt-4', 'claude-3-opus'). Required when type is LLM.
descriptionstringDetailed description of what the metric measures and when to use it
return_typestringData type of the metric's return value
child_metricsarrayList of child metrics with weights for composite metrics. Required when type is 'COMPOSITE'.
model_providerstringModel provider for LLM-evaluated metrics (e.g., 'openai', 'anthropic'). Required when type is LLM.
dataRequiredData from the action execution
errorstringError if any occurred during the execution of the action
successfulRequiredbooleanWhether or not the action execution was successful or not
API key connection. Risk level 2 of 5.
Free Action
No published Skills explicitly reference this Action yet.