Skip to Content
Troubleshooting

Troubleshooting

Common issues and solutions when using Lamdis.

Test Runs

Run stuck in “queued”

Cause: The test runner service is not processing the queue.

Solutions:

  • Check that the lamdis-runs service is running and accessible
  • Verify the connection between the dashboard and the runner
  • For self-hosted: ensure lamdis-runs container is healthy

View your runs → 

Judge returns errors

Cause: AWS Bedrock credentials or permissions are misconfigured.

Solutions:

  • Verify your AWS credentials have bedrock:InvokeModel permission
  • Check the BEDROCK_MODEL_ID environment variable is set correctly
  • Ensure the Bedrock model is available in your AWS region

Connection timeout

Cause: Your assistant endpoint is not reachable.

Solutions:

  • Verify your assistant’s URL is correct in the connection settings
  • Check firewall rules allow traffic from Lamdis
  • Test the endpoint manually with curl or Postman
  • Ensure your assistant is running and healthy

Manage connections → 

All tests fail immediately

Cause: Setup or connection issue preventing test execution.

Solutions:

  • Check the Setup configuration for errors
  • Verify environment variables are set correctly
  • Review the run detail page for specific error messages
  • Ensure your assistant responds in the expected format

Authentication

401 Unauthorized

Cause: Invalid or missing API key.

Solutions:

  • Verify your API key is valid and not expired
  • Check the key is included in the Authorization: Bearer <key> header
  • Regenerate the key if compromised

Manage API keys → 

403 Forbidden

Cause: Insufficient permissions for the requested action.

Solutions:

  • Verify your role has the required permissions
  • Contact an org admin to adjust your role
  • Check if the resource belongs to your organization

View roles → 

Results & Scoring

No results showing

Cause: Run hasn’t completed or encountered an error.

Solutions:

  • Confirm the run completed by checking its status
  • Look at the run detail page for errors
  • Check if the test suite contains any tests

View results → 

Judge scores all 0

Cause: Rubric doesn’t match assistant responses or is unclear.

Solutions:

  • Review your rubric for clarity and specificity
  • Ensure the rubric describes what “good” looks like
  • Check that your threshold (e.g., 0.75) is appropriate
  • Test with a simpler rubric to isolate the issue

Inconsistent judge scores

Cause: Ambiguous rubric or assistant responses vary significantly.

Solutions:

  • Make rubrics more specific with clear pass/fail criteria
  • Use the scope: transcript option to evaluate full conversations
  • Consider lowering the threshold for subjective evaluations

Connections & Integrations

Variable not found

Cause: Referenced variable doesn’t exist or has a typo.

Solutions:

  • Check variable names are UPPER_SNAKE_CASE
  • Verify the variable exists in your environment
  • Use the exact syntax: {{variables.YOUR_KEY_NAME}}

Manage variables → 

Action fails with 401/403

Cause: Connection credentials are invalid or expired.

Solutions:

  • Update the connection’s API key or OAuth credentials
  • Check if the external API key has expired
  • Verify the connection’s auth configuration

Manage connections → 

CI/CD Integration

Runs not triggering from CI

Cause: API authentication or endpoint misconfiguration.

Solutions:

  • Verify LAMDIS_API_KEY is set in CI secrets
  • Check the API URL is correct (https://api.lamdis.ai for hosted)
  • Ensure the API key has runs:execute permission

Can’t retrieve run results

Cause: Run ID is incorrect or key lacks read permission.

Solutions:

  • Verify the run ID from the trigger response
  • Ensure API key has runs:read permission
  • Check the run exists and belongs to your org

Self-Hosted Issues

lamdis-runs can’t connect to database

Cause: Database URL or credentials are wrong.

Solutions:

  • Verify MONGO_URL or DATABASE_URL is correct
  • Check database is accessible from the runner container
  • Ensure database user has required permissions

Bedrock calls fail

Cause: AWS credentials or region misconfigured.

Solutions:

  • Set AWS_REGION to a region with Bedrock access
  • Verify AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
  • Check IAM permissions include bedrock:InvokeModel
  • Ensure the model ID is valid for your region

Getting Help

If you can’t resolve an issue:

  1. Check the FAQ for common questions
  2. Review the specific feature documentation
  3. Email support@lamdis.ai with:
    • Error message or code
    • Steps to reproduce
    • Relevant configuration (without secrets)
Last updated on
Tests