​

External API

The External API endpoint allows you to retrieve JSON data from your saved API configurations. This endpoint returns data in your specified format, either from cache or freshly scraped from the source webpage.

​

Endpoint

GET https://api.pulpminer.com/external/{apiId}

​

Headers

​

Path Parameters

​

Response

{
  "data": {
    // Your structured JSON data
    "title": "Example Page Title",
    "content": "Main content of the page...",
    "metadata": {
      "author": "John Doe",
      "date": "2024-03-20"
    }
  },
  "errors": null
}

​

Response Fields

​

Error Responses

​

Invalid API Key

{
  "data": null,
  "errors": [
    {
      "message": "invalid key"
    }
  ]
}

​

Invalid API ID

{
  "data": null,
  "errors": [
    {
      "message": "no data"
    }
  ]
}

​

No Credits

{
  "data": null,
  "errors": [
    {
      "message": "No credits left ⚠. Please buy more credits"
    }
  ]
}

​

Server Error

{
  "data": null,
  "errors": [
    {
      "message": "something went wrong"
    }
  ]
}

​

Example Usage

​

Using cURL

curl -H "apikey: YOUR_API_KEY" https://api.pulpminer.com/external/123

​

Using JavaScript/Node.js

const response = await fetch('https://api.pulpminer.com/external/123', {
  headers: {
    'apikey': 'YOUR_API_KEY'
  }
});

const data = await response.json();

​

Using Python

import requests

headers = {
    'apikey': 'YOUR_API_KEY'
}

response = requests.get(
    'https://api.pulpminer.com/external/123',
    headers=headers
)
data = response.json()

​

Caching Behavior

The endpoint’s caching behavior depends on your API configuration:

​

Cache Enabled

• Returns cached data if available and less than 15 minutes old
• Automatically refreshes cache in the background if data is old
• Ensures fast response times while maintaining data freshness

​

Cache Disabled

• Always fetches fresh data from the source webpage
• May have slightly longer response times
• Guarantees up-to-date information

​

Notes

• This endpoint uses 2 credits per request
• Response format matches your saved API configuration
• Consider enabling caching for better performance
• Monitor your credit usage to avoid interruptions

​

Best Practices

1. Use Appropriate Caching

   • Enable caching for static or slowly changing data
   • Disable caching for real-time data requirements
   • Consider your application’s freshness vs. performance needs

2. Handle Errors

   • Implement proper error handling
   • Use retry logic with exponential backoff
   • Monitor for recurring errors

3. Optimize Usage

   • Cache responses on your end when possible
   • Batch related requests together
   • Implement rate limiting in your application

​

Rate Limiting

• 2 credits per API request
• Requests are processed in order
• Consider implementing backoff strategies for retries

​

Monitoring

Monitor your API usage through:

1. Last fetched timestamp in the API dashboard
2. Credit usage tracking
3. Response times and error rates

​

Need Help?

If you encounter issues:

1. Verify your API key and API ID
2. Check your available credits
3. Contact support at hello@pulpminer.com