​
Caching

Pulpminer’s caching system helps you optimize performance and reduce costs by storing and reusing webpage data. This guide explains how caching works and how to use it effectively.

​
How Caching Works

When you enable caching for an API endpoint:
  1. First Request
    • Data is fetched from the source webpage
    • The JSON response is stored in cache
    • The timestamp is recorded
  2. Subsequent Requests
    • If cached data is less than 15 minutes old:
      • Returns cached data immediately
    • If cached data is older than 15 minutes:
      • Returns cached data
      • Triggers a background refresh
      • Updates cache for future requests

​
Enabling Cache

You can enable caching when saving a new API:
  1. Generate your JSON from a webpage
  2. Review and customize the JSON structure
  3. Check the “Cache Content” toggle before saving
  4. Click “Save as API”

​
Cache Settings

SettingDescription
EnabledUses cached data with background refresh
DisabledAlways fetches fresh data

​
Benefits of Caching

​
1. Faster Response Times

  • Immediate responses from cache
  • No waiting for webpage scraping
  • Reduced latency

​
2. Cost Efficiency

  • Same credit cost per request
  • Better performance for the same price
  • More efficient resource usage

​
3. Reliability

  • Cached data available even if source is down
  • Background refresh reduces failures
  • Consistent response times

​
When to Use Caching

​
Best for:

  • Static content that rarely changes
  • High-traffic API endpoints
  • Non-time-critical data
  • Backup data availability
  • Real-time data requirements
  • Rapidly changing content
  • Time-sensitive information
  • Critical up-to-the-second updates

​
Cache Refresh

​
Automatic Refresh

  • Occurs when data is older than 15 minutes
  • Happens in the background
  • Doesn’t affect response time
  • Updates cache for next request

​
Manual Refresh

  • Delete and recreate the API endpoint
  • Forces a fresh data fetch
  • Resets the cache timer

​
Monitoring Cache Status

In your API dashboard, you can see:
  • Last fetch time for each endpoint
  • Cache status (enabled/disabled)
  • Next refresh time

​
Best Practices

  1. Choose Wisely
    • Enable caching for stable content
    • Disable for real-time requirements
    • Consider your use case needs
  2. Monitor Performance
    • Watch response times
    • Check data freshness
    • Adjust settings as needed
  3. Optimize Usage
    • Group similar data needs
    • Use consistent cache settings
    • Plan for scale

​
Troubleshooting

​
Common Issues

  1. Stale Data
    • Check if cache is enabled
    • Verify last fetch time
    • Consider disabling cache
  2. Slow Responses
    • Confirm cache status
    • Check source webpage
    • Monitor system status
  3. Cache Not Updating
    • Verify 15-minute threshold
    • Check for errors
    • Contact support

​
Need Help?

If you’re having caching issues:
  1. Review your cache settings
  2. Check the API dashboard
  3. Contact support at hello@pulpminer.com