ballerinax/webscraping.ai Ballerina library

2.0.0

Overview

WebScraping.AI is a web scraping API that provides LLM-powered tools, Chromium JavaScript rendering, rotating proxies and built-in HTML parsing. This connector lets Ballerina applications fetch page HTML and Markdown text, select page areas with CSS selectors, ask questions about a page, extract structured fields, run search engine queries and read structured data from supported sites. It supports version 3.2.2 of the WebScraping.AI API.

Key features

  • Retrieve page HTML or clean Markdown text, with JavaScript rendering and proxy options
  • Select one or more page areas by CSS selector
  • Ask an LLM questions about a page and extract structured fields from it
  • Search engine results (SERP) and structured data for supported sites
  • Check your remaining API credits and concurrency

Setup guide

To use the WebScraping.AI connector, you need an API key.

  1. Create an account at webscraping.ai and sign in.

  2. Open the dashboard and copy your API key.

  3. Store the key in a Config.toml file in your project, for example:

    Copy
    apiKey = "<YOUR_API_KEY>"

Quickstart

To use the webscraping.ai connector in your Ballerina application, update the .bal file as follows:

Step 1: Import the module

Copy
import ballerinax/webscraping.ai;

Step 2: Instantiate a new connector

Copy
configurable string apiKey = ?;

final ai:Client scraper = check new ({apiKey});

Step 3: Invoke the connector operation

Copy
public function main() returns error? {
    ai:Account _ = check scraper->getAccount();
}

Step 4: Run the Ballerina application

Copy
bal run

Examples

The WebScraping.AI connector provides practical examples illustrating usage in various scenarios. Explore these examples, covering the following use cases:

  1. Page content summarizer - Check the account quota, convert a web page to Markdown and ask an LLM a question about it.
  2. Search result extractor - Run a search query and extract a structured field from the top result.

Import

import ballerinax/webscraping.ai;Copy

Other versions

See more...

Metadata

Released date: about 15 hours ago

Version: 2.0.0

License: Apache-2.0


Security

Direct vulnerabilities: No known direct vulnerabilities

Indirect vulnerabilities: 

Checked on 2026-11-20 via OSV.dev


Compatibility

Platform: any

Ballerina version: 2201.12.0

GraalVM compatible: Yes


Pull count

Total: 60

Current verison: 0


Weekly downloads


Source repository


Keywords

Vendor/WebScraping.AI

Area/Developer Tools

Type/Connector


Contributors