How to use a screenshot API
When you add your first website inside webmaster.ninja, you will swiftly see a website screenshot. We managed to do that with a screenshot API, and this is the story of how we did it.
In the old version of our Website Manager the website screenshots were smaller and buggy. But, as we improved our Screenshot API, so have the results we saw in our list of websites.
New website manager with new Screenshot API
Since that version, we made a lot of changes both on our Website Manager, and the Screenshot API. I think that the results speak for themself and that the picture is, indeed, worth a thousand words. As you can see, the design is completely different, and all the screenshots are complete.
So, before we get to the nitty-gritty of how we implemented the Screenshot API and all the code examples that you can copy-paste, here’s an image of how the end result looks like. The screenshot below only shows the part with website cards, not the entire dashboard.
How to get access to a Screenshot API
Let’s start at the beginning. You need to find a screenshot API provider. Our parent company WhoAPI Inc. has one, so it was a no-brainer for us. They are active since 2011 (over a decade) which is important for an API provider. You don’t want to integrate an API, and then next year the API shuts down, and then you have to find another one.
Also, you don’t want the API documentation to constantly change! You need boring stability so that you don’t have to make constant updates to your code! But, you also want the API provider to be flexible and add features that your application requires. This we highlighted in the screenshots above. We weren’t happy with the results at first, and then after adding some features like “screenshot delay” we were able to get a great result.
Getting access to a Screenshot API is simple. You just register for an account on WhoAPI, and you instantly get an API key. This key will be used in every API request you make.
Complimentary 10,000 requests
At the moment, over at WhoAPI, they are celebrating their 10 year anniversary, so from now on, every new client gets 10,000 requests for free. This should be more than enough to get your application up and running. Once you have your API key, making your first API request is simple, fast, and easy. You can literally open the URL in your browser. Why? Well because it’s using the HTTP or HTTPS protocol!
A REST API (also known as RESTful API) is an application programming interface (API or web API) that conforms to the constraints of REST architectural style and allows for interaction with RESTful web services. REST stands for representational state transfer and was created by computer scientist Roy Fielding.
When a client request is made via a RESTful API, it transfers a representation of the state of the resource to the requester or endpoint. This information, or representation, is delivered in one of several formats via HTTP: JSON (Javascript Object Notation), HTML, XLT, Python, PHP, or plain text. JSON is the most generally popular programming language to use because, despite its name, it’s language-agnostic, as well as readable by both humans and machines.
Source: https://www.redhat.com/en/topics/api/what-is-a-rest-api
Making your first Screenshot API request
The URL you are “hitting” or requesting will look something like this:
https://api.whoapi.com/?domain=whoapi.com&r=screenshot&apikey=demokey&process=thumb
As a matter of fact, you can enter this exact URL in your own browser and see what happens. You will undoubtedly get a JSON response. Since we already have an API key, we will replace the “demokey” with our API key. An API key is just a long stream of letters and numbers that’s unique.
Once you make a request you will get a response that looks like this in your browser.
{"status":"0","full_size":"http:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/tmpfykDFs6j4V8UTaatHI.png","thumbnail":"http:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/rX2iU1g-CP75eyNTVVjTS.png","expires":"2021-09-10 10:24 GMT","size":"1366x768","full_size_https":"https:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/tmpfykDFs6j4V8UTaatHI.png","thumbnail_https":"https:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/rX2iU1g-CP75eyNTVVjTS.png","requests_available":471,"status_desc":"Successfully processed"}As you can see, we still have a long way to go, because this is not very useful to us. We want to be able to display a website screenshot inside of our application. So how do we get there? In the JSON response code you see above, there’s a URL. That’s where the website screenshot is stored.
https://storage.googleapis.com/whoapi-storage/screenshots/tmpfykDFs6j4V8UTaatHI.pngWe can either load the image from there (until the date of expiry), or we can store the image locally, and then load it from our local database. This will largely depend on what your application does, and how often do the website screenshots have to be updated.
In webmaster.ninja case, we want the user to have a fresh screenshot as often as possible. That way, the website investor can quickly see something suspicious without opening the website.
How to create website screenshots using Screenshot API
Our background service among other tasks makes the request to WhoAPI Screenshot API. The code is in PHP language. We use the following request parameters when generating domain thumbnails:
$screenshot_result = request_whoapi(
// Url for the screenshot is the main page of the domain
$domain_name,
// API type is Screenshot
'r=screenshot'
// We need thumbnail only (we do not need the full page with scroll)
.'&process=thumb'
// We will wait additional 3 seconds after the document is loaded,
// this will ensure that all the website animations are started.
.'&delay=3000'
// We need a big thumbnail with size of 582*330 pixels
.'&thumbwidth=582'
.'&thumbheight=330'
// Let's use second popular resolution (according StatCounter Worlwide stats),
// it is small enough to see all the website details in a thumbnail
.'&resolution=1366x768'
)
The request_function used in the example above looks like this:
/**
* Make request to WhoAPI API
*
* @param string Domain name
* @param string Request parameters
* @return array API reply
*/
function inner_request_api($domain_name, $request) {
/*
Make the request using cURL library
*/
{
$url = "https://api.whoapi.com/"
."?domain=".$domain_name
."&".$request
."&apikey=".API_KEY;
$ch = curl_init($url);
// We need the result
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// We need to fail on any HTTP error
curl_setopt($ch, CURLOPT_FAILONERROR, true);
// Execute the request and get the results
$response = curl_exec($ch);
// Get error (if any)
$error = curl_error($ch);
// Close the cURL handle
curl_close($ch);
}
/*
Validate the result
*/
{
// cURL returned HTTP-error? This is fatal. We can't continue.
if (!empty($error)) {
throw new APIException('Request failed, HTTP error: '.$error);
}
// API returned empty result? Something is wrong. We can't continue.
if (empty($response)) {
throw new APIException('Request failed, empty reply');
}
// Convert JSON string into PHP Array
$json = json_decode($response, true);
// Validate, is the JSON correct?
if (empty($json)
|| !is_array($json)
|| !isset($json['status'])
) {
throw new APIException('Request failed, reply format is invalid:'."\n".$response);
}
// API returned invalid processing status? This is fatal. We can't continue.
if ((int)$json['status'] !== 0) {
throw new APIException('Request failed, invalid status:'."\n".var_export($json, true));
}
}
// Return the resulting Array for the further processing and storing
return $json;
}
Additional explanation of the code
In brief, what this code does: it requests the API, checks the HTTP code of the reply, checks the reply contents, decodes the JSON string into an array, validates the processing status returned by the API, and passes the decoded result for further processing.
In Webmaster.Ninja we do not download the resulting thumbnails, we just use the direct links to images provided by API. This is possible because the WhoAPI stores images for 30 days and we update the thumbnails much frequently and the thumbnails will not expire. Also WhoAPI uses Google services for providing images, the output speed is very fast for any client location. This is very convenient for us and no additional work is required. We just make a request to the API, validate the result and store the thumbnail URL in our database.
I’ve been an online entrepreneur for more than a decade. Back in 2011, I sold my first small business. 500 Startups alumni. I love to read and write in every shape or form. Founder of WhoAPI and webmaster.ninja and website investor.
Should you monitor website performance?
I am just thinking if there’s a possibility of a movie script where the terrorist threatens to blow up the website if the website speed drops below 3 seconds. Probably not, but in ecommerce it certainly feels like this on some days. There’s a popular Amazon.com research that you probably heard of, and it goes like this; one second could cost Amazon $1.6 billion in sales. Holy macaroni, that’s a lot of zeroes for just 1 second. Internet users certainly got picky when 1 second determines if we stay and purchase, or hit the back button. From this, I conclude we should all monitor website performance, and here’s why.
Joke about Page load speed
There’s a joke I like to tell, and it relates to speed. Two men were walking in the forest, and they encounter a bear. As the bear started racing to them, one of the men quickly limbered up, tied his shoes and got ready to run as fast as he could.
To which the other man almost laughingly asked, certainly you don’t think you can outrun the bear? The other man replied; you are right! I can’t outrun the bear, but I will do my best to outrun you!
You be the judge if this same joke applies to website speed monitoring and page load speed.
Is website speed monitoring important?
I think the joke applies to load speed as well. If your website loads faster than the other, you win the game. It doesn’t have to load impossibly fast, it just has to load fast enough. I don’t want to sound like a slacker, I am a realist and pragmatist. You need to focus on so many stuff (recently I wrote about SSL expiration monitoring and domain expiration monitoring) that you have to choose your battles.
Where do you start? How high website speed monitoring should be on your list of priorities? In my opinion, it’s pretty high!
I love a slow website!
Nobody.
Have you ever heard anyone say they love when it takes forever to load a website? Back in 2005 until 2010 I’ve been building website for clients, and one complaint we often heard was the website is loading too slow. People just don’t like to wait, period!
That’s a dead giveaway you should focus on providing a great experience by having a fast loading website!
How is page load speed measured?
I won’t dive too deep and get to technical in this article (I may do it in the future), so I will provide a simplified version for now. When you access a website on your computer or mobile phone, your device downloads all the files from a server.
These files consist of images and videos (usually the heaviest files you download), text, programer’s code, and other parts. For example, some websites require the computer to download the font in order for the website to appear as it was intended by the designers.
Some time is spent on redirects (HTTP to HTTPS), and naturally, it takes a little time for a little DNS and whois magic to change the domain name like www.webmaster.ninja into an IP address (location of the server) like 206.81.7.80.
It’s also not the same if a website is being loaded in Chicago, USA, and the user’s computer is accessing the website from Florence, Italy.
What is the average website load speed?
According to research conducted by Backlinko, after analyzing over 5.2 million desktop and mobile pages; the average fully loaded website speed is 10.3 seconds on desktop and 27.3 seconds on mobile. So, can you imagine the excitement of your clients if your website loads in 3 seconds or less?
We’ll get into who else is excited when your website loads fast, but for now, let’s see how you can beat the average website load speed.
How to beat the average website load speed?
- Monitor website performance
- Optimize images and code
- Install a plugin
- Buy CDN, or at least a web hosting in the country users are accessing the website
- Use caching
- Reduce the number of website parts that need to load (if on WordPress, reduce the amount of plugins)
We will dive deeper some other time how you can improve website speed, for now let’s stay on topic of website speed monitoring.
Why did I put it on top of the list?
Why monitoring website performance is the first step to a fast website?
You could say that I put “monitor website performance” on top of the list because that’s exactly what we offer. And that is true, we even built a website speed test tool! We really care about speed, but we also have several other tools and monitors.
Here’s the real reason. How can you improve something if you don’t even know where you stand? You first need to do several tests, and then determine if your website loads above average, or below average.
Even better, if you set up constant website load speed monitoring you can check how the website responds over time. This leads to even better decisions on your end.
Website load speed monitoring in action
For example, in webmaster.ninja’s Website Manager, each time you log in, you can see the average homepage load speed. We will cover later the email notifications you get when your website takes a lot of time to load.
As you can see from the screenshot above, some of my websites are loading in record times. You could say that, like Amazon, I hate a slow website. My mission is to offer the best experience to my users, and website speed is one of them.
How often should webmasters test the website speed?
Well, if they do it manually, probably every day. Or, when they make a major change to their website. The problem is, sometimes the server is not performing as expected and the website slows down without you taking any part in it!
That’s where it’s super convenient to get an email that your page load speed decreased (the website is loading slowly), and you can be the first to react! You don’t want your clients telling you, you have a slow website. Sometimes, they will vote with their wallets and won’t tell you a thing.
This page load speed monitoring email notification is another example why starting to track your speed is the first step. When you see for yourself the moment website speed declines, and you see how often this happens, you instantly know if this should top your priority or not!
“What gets measured gets managed.”
Peter Drucker, consultant, educator, and author
Website performance tracking, why it’s a priority?
In case you are not sure if you should even focus on website performance tracking, why not do a few tests, and see how you stand. If you see your website loads slowly (10 seconds) for weeks, then you definitely need to add this to the list of your priorities.
If your website loads in under 3 seconds, you can focus in other areas! There are a lot of key performance indicators you can track in a website and page load speed is just one of them!
Is page load speed monitoring difficult to setup?
Are you kidding me? Setting up load speed monitoring is the easiest thing you will do today. Just add your domain name to our Website Manager, click “add website”, and you are done! It’s a one-click process, you set it and forget it!
In just a few days, you will have a beautiful graph (or not so beautiful if you have a slow website), and you will know if you are in good shape, or you have your work cut out for you.
Webmaster.Ninja speed test tool
If you just want to check your speed from multiple locations, you don’t have to setup a monitoring. You can access our website speed test tool, and do a one time check. It may not have all the bells and whistles other website speed test tools have, but it will do the job and measure the speed accurately.
Above you can see the webmaster.ninja website speed test tool in action. I tested one website under my account, and I am electrified as the website loads in top speed! With this tool you measure page load speed from multiple locations (3 in EU and 3 in US). If you think we should add 3 locations in Asia as well, please let us know.
Why is page speed important?
In case you need more benefits of a fast loading website, Googe decided to tip the needle for you. Back in 2018 Google announced that page speed is a ranking factor just like having an SSL! This means that if everything else is the same (remember the bear joke from the start of this long article?), a faster website will rank higher on the Google search engine results page.
This makes total sense! Google wants to show its users the fastest possible websites, not some grab-your-coffee-while-the-website-loads, type of a website! We also covered earlier that slow websites make the users close the website, or hit the back button. This action in return increases “the bounce rate” (when a user bounces off of a website) which is a bad signal to Google.
Free website performance monitoring
And now the best part. I explained how page speed is important, how easy it is to set it up. You could also see a glimpse of how complex and difficult it may be to keep tracking website speed over a period of time, from multiple locations in the world. So, you may reach a conclusion that this process could come with a very high cost?
How about, absolutely free? If you have one website, webmaster.ninja provides free website performance monitoring for your website. Actually, that’s not true. As you can tell by the start of the article, I love to make a poor attempt at telling a joke. Here’s how it goes. Free, rhymes with three, so in all seriousness, you can add up to three (3) websites, and track their speed. I promise we make better site speed monitoring tools, then we tell jokes.
If you need to add 4 website, or more, then you need to upgrade.
Page load speed is only a part of something bigger
Just as I left the “it’s free for up to three” announcement last, I have one last important thing to share with you. Page speed is a ranking factor, but Google is taking this to the next level. They want to include user experience in general, as a ranking factor! They are calling this the Core Web Vitals. Here’s what they say:
In the past several months, we’ve seen a median 70% increase in the number of users engaging with Lighthouse and PageSpeed Insights, and many site owners using Search Console’s Core Web Vitals report to identify opportunities for improvement. On Tuesday, November 10, 2020, we’re announcing that the page experience signals in ranking will roll out in May 2021. The new page experience signals combine Core Web Vitals with our existing search signals including mobile-friendliness, safe-browsing, HTTPS-security, and intrusive interstitial guidelines.
Bringing Page Experience to Google Search
What are Google Lighthouse and Google PageSpeed Insights?
PageSpeed Insights is a free tool built by Google that rates your website with a score, and in the process of doing that, it gives you advice (insights) on how to improve that score. Simplest way to learn more about PageSpeed Insight is to just test the tool by typing in your domain name, and let PSI do the work. It only takes about 30 seconds, depending on the website you are testing.
Sometimes the advice it gives isn’t particularly helpful if you are not a developer. Remove unused CSS, Remove unused JavaScript, Eliminate render-blocking resources, but nonetheless, it gives you a starting point if you set out to do something about website speed and loading time.
You can try Google PageSpeed Insights here.
Google Lighthouse on the other hand is the brain of the operation. Lighthouse is an open-source, automated tool for improving the quality of web pages. Pagespeed Insights (PSI) uses Lighthouse as its analysis engine.
Checking multiple pages at once
Now that we all agree we should check website page speed, let’s tackle another common problem. We don’t get traffic only to the homepage. Sometimes we have cornerstone content or a highly converting landing page as well… We want all these pages to load as fast as possible. So, how do we check multiple pages at once? Easy, through this free tool!
You just list all the URLs you want to check (your most popular ones, that we just talked about), and let this tool do the rest!
Yes, you should monitor website speed performance
Wrapping up with a clear answer to the question asked in the tittle of this article. Yes, we should all definitely monitor website speed performance, even more than some other website key performance indicators! Not just for the sake of Google and on page SEO, but more importantly, for the sake of our users!
If you care about the constant improvement of your website portfolio, efficiency and productivity, website load speed is a great way to practice kaizen and focus on the basics. If you don’t handle the basics, how can you handle more advanced challenges?
Next time you are frustrated with a slow website, think of this article, and then answer honestly if all your websites are loading as fast as they could?
I’ve been an online entrepreneur for more than a decade. Back in 2011, I sold my first small business. 500 Startups alumni. I love to read and write in every shape or form. Founder of WhoAPI and webmaster.ninja and website investor.