When you add your first website inside webmaster.ninja, you will swiftly see a website screenshot. We managed to do that with a screenshot API, and this is the story of how we did it.

In the old version of our Website Manager the website screenshots were smaller and buggy. But, as we improved our Screenshot API, so have the results we saw in our list of websites.

New website manager with new Screenshot API

Since that version, we made a lot of changes both on our Website Manager, and the Screenshot API. I think that the results speak for themself and that the picture is, indeed, worth a thousand words. As you can see, the design is completely different, and all the screenshots are complete.

So, before we get to the nitty-gritty of how we implemented the Screenshot API and all the code examples that you can copy-paste, here’s an image of how the end result looks like. The screenshot below only shows the part with website cards, not the entire dashboard.

How to get access to a Screenshot API

Let’s start at the beginning. You need to find a screenshot API provider. Our parent company WhoAPI Inc. has one, so it was a no-brainer for us. They are active since 2011 (over a decade) which is important for an API provider. You don’t want to integrate an API, and then next year the API shuts down, and then you have to find another one.

Also, you don’t want the API documentation to constantly change! You need boring stability so that you don’t have to make constant updates to your code! But, you also want the API provider to be flexible and add features that your application requires. This we highlighted in the screenshots above. We weren’t happy with the results at first, and then after adding some features like “screenshot delay” we were able to get a great result.

Getting access to a Screenshot API is simple. You just register for an account on WhoAPI, and you instantly get an API key. This key will be used in every API request you make.

Complimentary 10,000 requests

At the moment, over at WhoAPI, they are celebrating their 10 year anniversary, so from now on, every new client gets 10,000 requests for free. This should be more than enough to get your application up and running. Once you have your API key, making your first API request is simple, fast, and easy. You can literally open the URL in your browser. Why? Well because it’s using the HTTP or HTTPS protocol!

A REST API (also known as RESTful API) is an application programming interface (API or web API) that conforms to the constraints of REST architectural style and allows for interaction with RESTful web services. REST stands for representational state transfer and was created by computer scientist Roy Fielding.

When a client request is made via a RESTful API, it transfers a representation of the state of the resource to the requester or endpoint. This information, or representation, is delivered in one of several formats via HTTP: JSON (Javascript Object Notation), HTML, XLT, Python, PHP, or plain text. JSON is the most generally popular programming language to use because, despite its name, it’s language-agnostic, as well as readable by both humans and machines. 

Source: https://www.redhat.com/en/topics/api/what-is-a-rest-api

Making your first Screenshot API request

The URL you are “hitting” or requesting will look something like this:

https://api.whoapi.com/?domain=whoapi.com&r=screenshot&apikey=demokey&process=thumb

As a matter of fact, you can enter this exact URL in your own browser and see what happens. You will undoubtedly get a JSON response. Since we already have an API key, we will replace the “demokey” with our API key. An API key is just a long stream of letters and numbers that’s unique.

Once you make a request you will get a response that looks like this in your browser.

{"status":"0","full_size":"http:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/tmpfykDFs6j4V8UTaatHI.png","thumbnail":"http:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/rX2iU1g-CP75eyNTVVjTS.png","expires":"2021-09-10 10:24 GMT","size":"1366x768","full_size_https":"https:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/tmpfykDFs6j4V8UTaatHI.png","thumbnail_https":"https:\/\/storage.googleapis.com\/whoapi-storage\/screenshots\/rX2iU1g-CP75eyNTVVjTS.png","requests_available":471,"status_desc":"Successfully processed"}

As you can see, we still have a long way to go, because this is not very useful to us. We want to be able to display a website screenshot inside of our application. So how do we get there? In the JSON response code you see above, there’s a URL. That’s where the website screenshot is stored.

https://storage.googleapis.com/whoapi-storage/screenshots/tmpfykDFs6j4V8UTaatHI.png

We can either load the image from there (until the date of expiry), or we can store the image locally, and then load it from our local database. This will largely depend on what your application does, and how often do the website screenshots have to be updated.

In webmaster.ninja case, we want the user to have a fresh screenshot as often as possible. That way, the website investor can quickly see something suspicious without opening the website.

How to create website screenshots using Screenshot API

Our background service among other tasks makes the request to WhoAPI Screenshot API. The code is in PHP language. We use the following request parameters when generating domain thumbnails:

$screenshot_result = request_whoapi(
    // Url for the screenshot is the main page of the domain
    $domain_name,
    // API type is Screenshot
    'r=screenshot'
        // We need thumbnail only (we do not need the full page with scroll)
        .'&process=thumb'
        // We will wait additional 3 seconds after the document is loaded,
        // this will ensure that all the website animations are started.
        .'&delay=3000'
        // We need a big thumbnail with size of 582*330 pixels
        .'&thumbwidth=582'
        .'&thumbheight=330'
        // Let's use second popular resolution (according StatCounter Worlwide stats),
        // it is small enough to see all the website details in a thumbnail
        .'&resolution=1366x768'
)

The request_function used in the example above looks like this:

/**
* Make request to WhoAPI API
* 
* @param string Domain name
* @param string Request parameters
* @return array API reply
*/
function inner_request_api($domain_name, $request) {
    /*
        Make the request using cURL library
    */
    {
        $url = "https://api.whoapi.com/"
            ."?domain=".$domain_name
            ."&".$request
            ."&apikey=".API_KEY;
        
        $ch = curl_init($url);

        // We need the result
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

        // We need to fail on any HTTP error
        curl_setopt($ch, CURLOPT_FAILONERROR, true);
        
        // Execute the request and get the results
        $response = curl_exec($ch);

        // Get error (if any)
        $error = curl_error($ch);

        // Close the cURL handle
        curl_close($ch);
    }
    
    /*
        Validate the result
    */
    {
        // cURL returned HTTP-error? This is fatal. We can't continue.
        if (!empty($error)) {
            throw new APIException('Request failed, HTTP error: '.$error);
        }
        
        // API returned empty result? Something is wrong. We can't continue.
        if (empty($response)) {
            throw new APIException('Request failed, empty reply');
        }

        // Convert JSON string into PHP Array
        $json = json_decode($response, true);

        // Validate, is the JSON correct?
        if (empty($json)
            || !is_array($json)
            || !isset($json['status'])
        ) {
            throw new APIException('Request failed, reply format is invalid:'."\n".$response);
        }
        
        // API returned invalid processing status? This is fatal. We can't continue.
        if ((int)$json['status'] !== 0) {
            throw new APIException('Request failed, invalid status:'."\n".var_export($json, true));
        }
    }
    
    // Return the resulting Array for the further processing and storing
    return $json;
}

Additional explanation of the code

In brief, what this code does: it requests the API, checks the HTTP code of the reply, checks the reply contents, decodes the JSON string into an array, validates the processing status returned by the API, and passes the decoded result for further processing.

In Webmaster.Ninja we do not download the resulting thumbnails, we just use the direct links to images provided by API. This is possible because the WhoAPI stores images for 30 days and we update the thumbnails much frequently and the thumbnails will not expire. Also WhoAPI uses Google services for providing images, the output speed is very fast for any client location. This is very convenient for us and no additional work is required. We just make a request to the API, validate the result and store the thumbnail URL in our database.