Honeycomb

Why Use Honeycomb with Foyle

Honeycomb is often used to query and visualize observability data.

Using Foyle and RunMe you can

• Use AI to turn a high level intent into an actual Honeycomb query
• Capture that query as part of a playbook or incident report
• Create documents that capture the specific queries you are interested in rather than rely on static dashboards

How to integrate Honeycomb with RunMe

This section explains how to integrate executing Honeycomb queries from RunMe.

Honeycomb supports embedding queries directly in the URL. You can use this feature to define queries in your notebook and then generate a URL that can be opened in the browser to view the results.

Honeycomb queries are defined in JSON. A suggested pattern to define queries in your notebook are

1. Create a code cell which uses contains the query to be executed
   • To make it readable the recommended pattern is to treat it as a multi-line JSON string and write it to a file
2. Use the CLI hccli to generate the URL and open it in the browser
   • While the CLI allows passing the query as an argument this isn’t nearly as human readable as writing it and reading it from a temporary file Here’s an example code cell

cat << EOF > /tmp/query3.json
{
  "time_range": 604800,
  "granularity": 0,  
  "calculations": [
      {
          "op": "AVG",
            "column": "prediction.ok"
      }
  ],
  "filters": [
      {
          "column": "model.name",
          "op": "=",
          "value": "llama3"
      }
  ],
  "filter_combination": "AND",
  "havings": [],
  "limit": 1000
}
EOF
hccli querytourl --query-file=/tmp/query3.json --base-url=https://ui.honeycomb.io/YOURORG/environments/production --dataset=YOURDATASET --open=true

• This is a simple query to get the average of the prediction.ok column for the llama3 model for the last week
• Be sure to replace YOURORG and YOURDATASET with the appropriate values for your organization and dataset
  • You can determine base-url just by opening up any existing query in Honeycomb and copying the URL
• When you execute the cell, it will print the query and open it in your browser
• You can use the share query feature to encode the query in the URL

Training Foyle to be your Honeycomb Expert

To train Foyle to be your Honeycomb you follow these steps

1. Make sure you turned on learning by following the guide

2. Create a markdown cell expressing the high level intent you want to accomplish

3. Ask Foyle to generate a completion

   • The first time you do this Foyle will probably provide a widely inaccurate answer because it has no prior knowledge of your infrastructure
   • Edit one of the generated code cells to contain the Honeycomb query and hccli command
   • Execute the cell to generate the URL

4. Each time you ask for a completion and edit and execute the cell you are providing feedback to Foyle

   • Foyle uses the feedback to learn and improve the completions it generates

Important When providing feedback to Foyle its important to do so by editing a code cell that was generated by Foyle. If you create a new code cell Foyle won’t be able to learn from it. Only cells that were generated by Foyle are linked to the original query.

If you need help boostrapping some initial Honeycomb JSON queries you can use Honeycomb’s Share feature to generate the JSON from a query constructed in the UI.

Producing Reports

RunMe’s AutoSave Feature creates a markdown document that contains the output of the code cells. This is great for producing a report or writing a postmortem. When using this feature with Honeycomb you’ll want to capture the output of the query. There are a couple ways to do this

Permalinks

To generate permalinks, you just need to use can use start_time and end_time to specify a fixed time range for your queries (see Honeycomb query API). Since the hccli prints the output URL it will be saved in the session outputs generated by RunMe. You could also copy and past the URL into a markdown cell.

Grabbing a Screenshot

Unfortunately a Honeycomb enterprise plan is required to access query results and graphs via API. As a workaround, hccli supports grabbing a screenshot of the query results using browser automation.

You can do this as follows

1. Restart chrome with a debugging port open

   chrome --remote-debugging-port=9222
   

2. Login into Honeycomb

3. Add the --out-file=/path/to/file.png to hccli to specify a file to save it to

   hccli querytourl --query-file=/tmp/query3.json --base-url=https://ui.honeycomb.io/YOURORG/environments/production --dataset=YOURDATASET --open=true --out-file=/path/to/file.png
   

Warning Unfortunately this tends to be a bit brittle for the following reasons

• You need to restart chrome with remote debugging enabled

• hccli will use the most recent browser window so you need to make sure your most recent browser window is the one with your Honeycomb credentials. This may not be the case if you have different accounts logged into different chrome sessions

Reference

• Honeycomb template links