Datarobot Create Genai Evaluation Dataset Configurations

Pricing

Free

Unified payment

Not currently supported

• namestring
  Custom configuration name for the evaluation dataset (max 5000 characters). If omitted, DataRobot will auto-generate a name.
• datasetIdstring

  Required
  The evaluation dataset identifier (24-character hex string). Use DATAROBOT_LIST_DATASETS to find available datasets.
• useCaseIdstring

  Required
  The use case identifier (24-character hex string). This links the evaluation dataset to a specific use case.
• playgroundIdstring

  Required
  The playground identifier (24-character hex string). This links the evaluation to a specific GenAI playground.
• promptColumnNamestring

  Required
  The name of the dataset column containing prompt text. This column provides the input prompts for evaluation.
• correctnessEnabledboolean
  Enable legacy correctness evaluation. This flag controls whether to use DataRobot's built-in correctness metrics.
• isSyntheticDatasetboolean
  Indicates whether the dataset contains synthetic data (default: false). Set to true if using AI-generated evaluation data.
• responseColumnNamestring
  The name of the dataset column containing expected response text. Use this to compare actual LLM responses against ground truth.
• toolCallsColumnNamestring
  The name of the dataset column containing expected tool calls for agentic workflows. Use this for evaluating agents that call external tools.
• agentGoalsColumnNamestring
  The name of the dataset column containing expected agent goals for agentic workflows. Use this for evaluating multi-step agent behavior.

• data

  Required
  Data from the action execution
• errorstring
  Error if any occurred during the execution of the action
• successfulboolean

  Required
  Whether or not the action execution was successful or not

No verified related use cases are linked yet.

No published Skills explicitly reference this Action yet.