# News API Documentation

## Overview
The News API provides comprehensive news aggregation from multiple sources including NewsAPI, Currents API, and RSS feeds. It supports categorization, search, and real-time news updates across various topics.

## Features
- **Multi-source aggregation**: NewsAPI, Currents API, and RSS feeds
- **Smart categorization**: 14 predefined categories with auto-categorization
- **Advanced search**: Full-text search across titles, descriptions, and content
- **Caching**: Built-in caching for improved performance
- **Rate limiting**: Respects API limits and implements retry logic
- **Deduplication**: Prevents duplicate articles across sources

## Categories
1. **Career and Skills** - Job opportunities, career development, skills
2. **Technology** - Tech news, software, programming, AI
3. **Startups and Innovation** - Entrepreneurship, funding, innovation
4. **Education** - Academic news, learning resources, student updates
5. **Sports** - Sports news, matches, athletic achievements
6. **Movies and Entertainment** - Entertainment industry, films, music
7. **Business and Economy** - Business news, financial markets, economy
8. **Politics** - Political news, government updates, policy changes
9. **Astrology** - Horoscopes, astrological insights, cosmic updates
10. **Health and Wellness** - Medical news, health tips, wellness
11. **Environment and Climate** - Environmental news, climate change, sustainability
12. **Art and Culture** - Cultural events, art exhibitions, creative expressions
13. **Knowledge Bytes and Facts** - Interesting facts, scientific discoveries
14. **Jobs** - Job opportunities, recruitment news, employment updates

## API Endpoints

### Get All News
```
GET /api/news
```
**Query Parameters:**
- `category` (string): Filter by category
- `source` (string): Filter by source (newsapi, currents, rss)
- `page` (number): Page number (default: 1)
- `limit` (number): Articles per page (default: 20)
- `sortBy` (string): Sort field (default: publishedAt)
- `sortOrder` (string): Sort order - 'asc' or 'desc' (default: desc)

**Example:**
```bash
GET /api/news?category=technology&page=1&limit=10&sortBy=publishedAt&sortOrder=desc
```

### Get News by Category
```
GET /api/news/category/:category
```
**Parameters:**
- `category`: Category slug (e.g., 'technology', 'career-and-skills')

**Query Parameters:**
- `page` (number): Page number (default: 1)
- `limit` (number): Articles per page (default: 20)

### Get News by Source
```
GET /api/news/source/:source
```
**Parameters:**
- `source`: Source name (newsapi, currents, rss)

**Query Parameters:**
- `page` (number): Page number (default: 1)
- `limit` (number): Articles per page (default: 20)

### Search News
```
GET /api/news/search
```
**Query Parameters:**
- `q` (string): Search query (required, min 2 characters)
- `page` (number): Page number (default: 1)
- `limit` (number): Articles per page (default: 20)

**Example:**
```bash
GET /api/news/search?q=artificial intelligence&page=1&limit=10
```

### Get Single News Article
```
GET /api/news/:id
```
**Parameters:**
- `id`: MongoDB ObjectId of the news article

### Get News Categories
```
GET /api/news/categories
```
Returns all available news categories with article counts.

### Get News Sources
```
GET /api/news/sources
```
Returns all configured news sources with their status.

### Get Latest News
```
GET /api/news/latest
```
**Query Parameters:**
- `limit` (number): Number of articles (default: 20)

### Get Trending News
```
GET /api/news/trending
```
**Query Parameters:**
- `limit` (number): Number of articles (default: 10)
- `days` (number): Number of days to look back (default: 7)

### Get News Statistics
```
GET /api/news/stats
```
Returns comprehensive statistics about news articles, categories, and sources.

### Refresh News
```
POST /api/news/refresh
```
**Body:**
```json
{
  "categories": ["technology", "business"] // Optional: specific categories to refresh
}
```
Fetches fresh news from all configured sources.

### Clear Cache
```
POST /api/news/cache/clear
```
Clears the news cache to force fresh data retrieval.

## Response Format

### Success Response
```json
{
  "success": true,
  "data": {
    "news": [...],
    "pagination": {
      "page": 1,
      "limit": 20,
      "total": 100,
      "pages": 5
    }
  }
}
```

### Error Response
```json
{
  "success": false,
  "message": "Error description",
  "error": "Detailed error message"
}
```

## News Article Schema
```json
{
  "_id": "ObjectId",
  "title": "Article title",
  "description": "Article description",
  "content": "Full article content",
  "url": "Original article URL",
  "imageUrl": "Article image URL",
  "source": "newsapi|currents|rss",
  "sourceName": "Source name",
  "category": "category-slug",
  "publishedAt": "2024-01-01T00:00:00.000Z",
  "fetchedAt": "2024-01-01T00:00:00.000Z",
  "author": "Author name",
  "tags": ["tag1", "tag2"],
  "isActive": true,
  "viewCount": 0,
  "hash": "unique-hash",
  "metaDescription": "SEO description",
  "wordCount": 500,
  "readingTime": 3,
  "createdAt": "2024-01-01T00:00:00.000Z",
  "updatedAt": "2024-01-01T00:00:00.000Z"
}
```

## Setup and Installation

### 1. Install Dependencies
```bash
npm install rss-parser
```

### 2. Initialize News Data
```bash
npm run init:news
```

### 3. Start the Server
```bash
npm run dev
```

### 4. Test the API
```bash
npm run test:news
```

## Configuration

### Environment Variables
- `MONGODB_URI`: MongoDB connection string
- `NEWSAPI_KEY`: NewsAPI key (default: 4a039be72844448a9adcd837fb920bbc)
- `CURRENTS_API_KEY`: Currents API key (default: OwUf8-lpvINu6cfU20IfxPNxlEO9oYYZnch-3l-k6afTJ0Xu)

### News Sources Configuration
Sources are configured in the database and can be managed through the NewsSource model:
- Rate limiting settings
- API keys and endpoints
- Retry logic
- Health monitoring

## Caching Strategy
- **In-memory caching**: 5-minute cache for frequently accessed data
- **Database indexing**: Optimized indexes for fast queries
- **Cache invalidation**: Automatic cache clearing on news refresh

## Rate Limiting
- **NewsAPI**: 100 requests/minute, 1000 requests/hour
- **Currents API**: 60 requests/minute, 500 requests/hour
- **RSS Feeds**: No rate limits, but respectful delays between requests

## Error Handling
- **API failures**: Graceful fallback to other sources
- **Rate limit exceeded**: Automatic retry with exponential backoff
- **Network timeouts**: Configurable timeout settings
- **Invalid responses**: Data validation and sanitization

## Monitoring and Analytics
- **Source health**: Real-time monitoring of API health
- **Success rates**: Track API success/failure rates
- **Response times**: Monitor API response times
- **Article statistics**: Track article counts by category and source

## Security Features
- **Input validation**: All inputs are validated and sanitized
- **SQL injection prevention**: MongoDB injection protection
- **XSS protection**: Content sanitization
- **Rate limiting**: Prevent abuse and respect API limits

## Performance Optimization
- **Database indexing**: Optimized indexes for common queries
- **Pagination**: Efficient pagination for large datasets
- **Caching**: Multi-level caching strategy
- **Compression**: Response compression for faster transfers

## Troubleshooting

### Common Issues
1. **No news articles**: Run `npm run init:news` to initialize categories
2. **API errors**: Check API keys and rate limits
3. **Slow responses**: Clear cache with `/api/news/cache/clear`
4. **Missing categories**: Verify category slugs match exactly

### Debug Mode
Enable debug logging by setting `NODE_ENV=development` in your environment.

## Contributing
1. Follow the existing code structure
2. Add proper error handling
3. Include tests for new features
4. Update documentation

## License
This API is part of the Recon App project.
