One of the first tasks I was assigned to at my new workplace ( ✨Deep BlueDot ✨) was to find a way to fetch console logs (info, errors, warnings, asserts etc) and HAR network logs(Network requests and responses) of a webpage and access it through chrome extension.
Devign - A Data-based documentation tool
Super easy browser extension ready for instant disposal. Build a robust team library, communicate based on single source of data-point, and report issues with the right context. Launch more perfect products, faster. Loved by teams that transform good ideas into great products. Team productivity is all about being on the same page Minimize multiple channel communications.
Devign provides the essential context and data for developers such as console logs, network logs, dev environment, user flow, user footprints, and more—to resolve the issue.
This article shines light upon how the Devign chrome extension fetches the network log information (HAR - HTTP Archive) using Chrome’s webRequest API which is useful for debugging.
HAR logs capture all network information including personal information such as cookies or passwords. Be careful about whom you share the log information with!
How the Chrome Extension webRequest API works
manifest.jsonmust include webRequest permission in order to access webRequests API
The web request API presents an abstraction of the network stack to the extension.
Internally, one URL request can be split into several HTTP requests (for example to fetch individual byte ranges from a large file) or can be handled by the network stack without communicating with the network.
Background (in version 3, referred to as
What service workers do: listen for and respond to events in order to enhance the end user's experience.
webRequest API events used:
Fired before sending an HTTP request, once the request headers are available. This may occur after a TCP connection is made to the server, but before any HTTP data is sent.
chrome.webRequest.onBeforeSendHeaders.addListener( callback: function, filter: RequestFilter, extraInfoSpec?: OnBeforeSendHeadersOptions, )
Fired when HTTP response headers of a request have been received.
chrome.webRequest.onHeadersReceived.addListener( callback: function, filter: RequestFilter, extraInfoSpec?: OnHeadersReceivedOptions, )
Note: The following example stores the network log data in an array but Devign stores the data in a queue.
- Ignoring requests initiated by the extension with :
- Storing requestTimeStamp and responseTimeStamp separately
- Example of the Request, Response Header structures (Variable - depends on the individual request type)
src/background/api.ts - Post the collected network log data to backend server
This is the part of the extension which can access the Document Object Model (DOM) of the webpage and tells the service workers when to start collecting the network log data.
Devign extension uses a toggle button to decide whether to collect logs or not. When toggled on, the content script sends a message to the background to capture the network logs using the following code:
Here’s what it looks like in the Dashboard
Once the Network log data has been stored in the backend server, it can be parsed and displayed as shown below.