Frequently Asked Questions

Keylime Toolbox - General

How flexible is Keylime Toolbox if some data is unavailable?

It's rare that data isn't available from the data sources Keylime Toolbox pulls from. However, if data isn't available for a particular day from Google Search Console or Google Analytics, Keylime Toolbox recovers any missing data the next day.

How accurate is the data?

Keylime Toolbox imports data directly from Google Search Console and Google Analytics. Both sources provide incomplete query coverage (although Keylime Toolbox uses several methods to uncover as much query data as possible and estimate total traffic for each segment). Keylime Toolbox always includes information on the number of provided queries for a segment and indicates when totals listed are estimates.

Does Keylime Toolbox show trended ranking data for each query?

Keylime Toolbox is built around segments to provide insight on the big picture for your site.

Should I be concerned about the privacy of my data with Keylime Toolbox?

Keylime Toolbox is built with privacy in mind. No personally identifiable customer information is stored except your name when you create an account (Google Search Console provides only anonymous aggregated data). Your data is hosted on secure Google and Amazon Web Services systems and all data is behind a secure login.

Our site is small and we don’t get much search traffic. Will Keylime Toolbox be able to provide value to us?

Keylime Toolbox is valuable as long as the site is getting enough search traffic for Google Analytics and Google Search Console to report it.

Our site is huge and we get tons of search traffic. Google Analytics only provides sampled data. Will Keylime Toolbox be able to provide value to us?

Keylime Toolbox was built to scale and can process any number of queries. Keylime Toolbox pulls data directly from the Google Analytics API, which generally avoids any sampling issues. Google Search Console provides only top queries, but by aggregating and de-duplicating subfolder data, Keylime Toolbox can provide substantially more detail than is available in Google Search Console directly. In addition, since Keylime Toolbox provides data and trends based on segments rather than individual queries (and estimates total traffic for each segment), the reports provide significant big picture trends and insights.

Keylime Toolbox - Google Search Console

How granular is the reporting data?

Keylime Toolbox imports data from Google Search Console and Google Analytics daily. You can choose to view data by day, week, month, or year.

Keylime Toolbox is based in part on Google Search Console data. How is Keylime Toolbox different from logging into Google Search Console directly?

Keylime Toolbox aggregates and de-duplicates query data from all subfolders to provide substantially more query and visit detail than is available in Google Search Console directly. Keylime Toolbox also integrates Google Analytics data for greater combined insight. Keylime Toolbox stores historical data over time to provide long-term trends and also processes the data in a number of ways not available in the Google Search Console interface. You can view click through rates by segment by position, trends of each SEO metric over time (also by segment), and understand what topic areas perform best.

Is there a limit on the number of segments?

There is no limit on the number of segments. The right number of segments depends on your site’s topics and audiences. Too few segments and you won’t be able to see patterns. Too many segments and the data will be too granular.

Google Search Console seems to have a lag in reporting. How does Keylime Toolbox handle this?

Keylime Toolbox imports the most recent data available from Google Search Console each night. What you see in Keylime Toolbox is the latest data available from Google Search Console (which is typically about two days behind). 

How are segments created?

Keylime Toolbox segments queries to provide insights based on patterns. Patterns can be as simple as a few characters to match or as complex as a regular expression.

Keylime Toolbox - Server Log Analysis

How does Keylime Toolbox access our server logs?

We provide instructions for uploading log files to a private Amazon Web Services S3 bucket, which Keylime Toolbox then reads from. We recommend setting up a script that uploads log files daily.

Do we have to upload all server logs or will a sample from one server be sufficient?

We recommend providing full log files if possible. This enables more comprehensive diagnostics. For instance, if some URLs are not indexed, we can determine if they’ve ever been crawled and we can provide data such as how long it takes for a site to be completely re-crawled. If the site is load balancing, having logs from all servers helps us find misconfiguration issues with a single server that may not be evident. However, if we have only sampled logs, Keylime Toolbox can still provide substantial information, such as canonicalization issues, server errors, problems with redirects (302s instead of 301s for instance).

We can’t provide you with our server logs because they contain customer data. Do we have any alternatives?

Keylime Toolbox processes search engine bot data only, so you can exclude all customer data before uploading the log files. Our instructions provide details of writing a script that parses out only the data we need before uploading.

Our log files are huge! Is there anything we can do to make them smaller before we upload them?

As noted above, Keylime Toolbox processes search engine bot data only, so you can exclude all customer data before uploading the log files. Our instructions provide details of writing a script that parses out only the data we need before uploading. That alone should make the files significantly smaller. In addition, you can set your script to upload the logs at any interval you’d like. Rather than uploading an entire days’s worth at once, you can upload several times a day. You can also compress the files before uploading them.

How far behind does Keylime Toolbox report server log data?

Keylime Toolbox processes server log data each night, so by uploading daily, you’ll always see information from the previous day. Once processed historical data is always available.

Keylime Toolbox - International Support

What countries and languages does Keylime Toolbox support?

While the user interface is currently in English, Keylime Toolbox supports all languages and character sets supported in Google Search Console. Note that since the Google Search Console API currently only provides data across all countries, the query data Keylime Toolbox provides is also not country-specific.

What search engines does Keylime Toolbox provide crawl analytics for?

Our server log analytics shows data for all major search engines, including Google, Bing, Yahoo, Naver, Yandex, and Baidu.