renato97/MangaReader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

chore: clean unnecessary markdown files for CV sharing

Browse Source
This commit is contained in:
Renato97
2026-03-31 01:16:14 -03:00
parent 89cdb5468f
commit f2a6d682c6
23 changed files with 0 additions and 10513 deletions
Download Patch File Download Diff File Expand all files Collapse all files

197
backend/TEST_QUICK_START.md
View File

@@ -1,197 +0,0 @@
# Quick Start Guide: Integration Tests
## Prerequisites
```bash
# Install dependencies (if not already installed)
cd /home/ren/ios/MangaReader/backend
npm install
```
## Method 1: Using npm scripts (Recommended)
### Run individual tests:
```bash
# Terminal 1: Start server
npm start
# Terminal 2: Run VPS flow test
npm run test:vps
# Terminal 3: Run concurrent downloads test
npm run test:concurrent
```
### Clean up test data:
```bash
npm run test:clean
```
## Method 2: Using the test runner script
### Basic commands:
```bash
# Start server in background
./run-tests.sh start
# Check server status
./run-tests.sh status
# View server logs
./run-tests.sh logs
# Run VPS flow test
./run-tests.sh vps-flow
# Run concurrent downloads test
./run-tests.sh concurrent
# Run all tests
./run-tests.sh all
# Clean up test data
./run-tests.sh cleanup
# Stop server
./run-tests.sh stop
```
### Complete workflow (one command):
```bash
./run-tests.sh start && ./run-tests.sh all && ./run-tests.sh cleanup && ./run-tests.sh stop
```
## Method 3: Manual execution
```bash
# Terminal 1: Start server
node server.js
# Terminal 2: Run VPS flow test
node test-vps-flow.js
# Terminal 3: Run concurrent downloads test
node test-concurrent-downloads.js
```
## What Gets Tested
### VPS Flow Test (`test-vps-flow.js`)
- ✓ Server health check
- ✓ Chapter image scraping
- ✓ Download to VPS storage
- ✓ File verification
- ✓ Storage statistics
- ✓ Chapter deletion
- ✓ Complete cleanup
### Concurrent Downloads Test (`test-concurrent-downloads.js`)
- ✓ 5 chapters downloaded concurrently
- ✓ No race conditions
- ✓ No file corruption
- ✓ Independent manifests
- ✓ Concurrent deletion
- ✓ Thread-safe operations
## Expected Output
### Success:
```
✓ ALL TESTS PASSED
✓ No race conditions detected
✓ No file corruption found
✓ Storage handles concurrent access properly
```
### Test Results:
```
Total Tests: 11
Passed: 11
Failed: 0
```
## Troubleshooting
### Port already in use:
```bash
lsof -ti:3000 | xargs kill -9
```
### Server not responding:
```bash
# Check if server is running
./run-tests.sh status
# View logs
./run-tests.sh logs
```
### Clean everything and start fresh:
```bash
# Stop server
./run-tests.sh stop
# Clean test data
./run-tests.sh cleanup
# Remove logs
rm -rf logs/
# Start fresh
./run-tests.sh start
```
## Test Duration
- **VPS Flow Test**: ~2-3 minutes
- **Concurrent Test**: ~3-5 minutes
Total time: ~5-8 minutes for both tests
## Files Created
| File | Purpose |
|------|---------|
| `test-vps-flow.js` | End-to-end VPS flow tests |
| `test-concurrent-downloads.js` | Concurrent download tests |
| `run-tests.sh` | Test automation script |
| `TEST_README.md` | Detailed documentation |
| `TEST_QUICK_START.md` | This quick reference |
## Getting Help
```bash
# Show test runner help
./run-tests.sh help
# View detailed documentation
cat TEST_README.md
```
## Next Steps
After tests pass:
1. ✓ Verify storage directory structure
2. ✓ Check image quality in downloaded chapters
3. ✓ Monitor storage stats in production
4. ✓ Set up CI/CD integration (see TEST_README.md)
## Storage Location
Downloaded test chapters are stored in:
```
/home/ren/ios/MangaReader/storage/manga/one-piece_1695365223767/
├── chapter_787/
├── chapter_788/
├── chapter_789/
├── chapter_790/
└── chapter_791/
```
Each chapter contains:
- `page_001.jpg`, `page_002.jpg`, etc. - Downloaded images
- `manifest.json` - Chapter metadata and image list

246
backend/TEST_README.md
View File

@@ -1,246 +0,0 @@
# Integration Tests for MangaReader VPS Backend
This directory contains comprehensive integration tests for the complete VPS flow: iOS app → VPS backend → Storage → Images served back.
## Test Files
### 1. `test-vps-flow.js`
Tests the complete end-to-end flow of downloading and serving manga chapters.
**Test Coverage:**
- Server health check
- Chapter image scraping from source
- Download to VPS storage
- Storage verification
- Image file validation
- Image path retrieval
- Chapter listing
- Storage statistics
- Chapter deletion
- Post-deletion verification
- Storage stats update verification
**Usage:**
```bash
# Make sure the server is running first
node server.js &
# In another terminal, run the test
node test-vps-flow.js
```
**Expected Output:**
- Color-coded test progress
- Detailed assertions with success/failure indicators
- Storage statistics
- Final summary with pass/fail counts
### 2. `test-concurrent-downloads.js`
Tests concurrent download operations to verify thread safety and data integrity.
**Test Coverage:**
- Pre-download cleanup
- Concurrent chapter downloads (5 chapters, max 3 concurrent)
- Post-download verification
- File integrity checks (no corruption, no missing files)
- Manifest independence verification
- Storage statistics accuracy
- Chapter listing functionality
- Concurrent deletion
- Complete cleanup verification
- Race condition detection
**Usage:**
```bash
# Make sure the server is running first
node server.js &
# In another terminal, run the test
node test-concurrent-downloads.js
```
**Expected Output:**
- Progress tracking for each operation
- Batch processing information
- Detailed integrity reports per chapter
- Summary of valid/missing/corrupted images
- Concurrent delete tracking
- Final summary with race condition analysis
## Test Configuration
Both tests use the following configuration:
```javascript
{
mangaSlug: 'one-piece_1695365223767',
chapters: [787, 788, 789, 790, 791], // For concurrent test
baseUrl: 'http://localhost:3000',
timeout: 120000-180000 // 2-3 minutes
}
```
You can modify these values in the test files if needed.
## Prerequisites
1. **Dependencies installed:**
```bash
npm install
```
2. **Server running on port 3000:**
```bash
node server.js
```
3. **Storage directory structure:**
The tests will automatically create the required storage structure:
```
/storage
/manga
/one-piece_1695365223767
/chapter_789
page_001.jpg
page_002.jpg
...
manifest.json
```
## Running All Tests
Run both test suites:
```bash
# Terminal 1: Start server
cd /home/ren/ios/MangaReader/backend
node server.js
# Terminal 2: Run VPS flow test
node test-vps-flow.js
# Terminal 3: Run concurrent downloads test
node test-concurrent-downloads.js
```
## Test Results
### Success Indicators
- ✓ Green checkmarks for passing assertions
- 🎉 "ALL TESTS PASSED!" message
- Exit code 0
### Failure Indicators
- ✗ Red X marks for failing assertions
- ❌ "SOME TESTS FAILED" message
- Detailed error messages
- Exit code 1
## Color Codes
The tests use color-coded output for easy reading:
- **Green**: Success/passing assertions
- **Red**: Errors/failing assertions
- **Blue**: Information messages
- **Cyan**: Test titles
- **Yellow**: Warnings
- **Magenta**: Operation tracking (concurrent tests)
## Cleanup
Tests automatically clean up after themselves:
- Delete test chapters from storage
- Remove temporary files
- Reset storage statistics
However, you can manually clean up:
```bash
# Remove all test data
rm -rf /home/ren/ios/MangaReader/storage/manga/one-piece_1695365223767
```
## Troubleshooting
### Server Not Responding
```
Error: Failed to fetch
```
**Solution:** Make sure the server is running on port 3000:
```bash
node server.js
```
### Chapter Already Exists
Tests will automatically clean up existing chapters. If you see warnings, that's normal behavior.
### Timeout Errors
If tests timeout, the scraper might be taking too long. You can:
1. Increase the timeout value in TEST_CONFIG
2. Check your internet connection
3. Verify the source website is accessible
### Port Already in Use
```
Error: listen EADDRINUSE: address already in use :::3000
```
**Solution:** Kill the existing process:
```bash
lsof -ti:3000 | xargs kill -9
```
## Test Coverage Summary
| Feature | VPS Flow Test | Concurrent Test |
|---------|---------------|-----------------|
| Server Health | ✓ | - |
| Image Scraping | ✓ | ✓ |
| Download to Storage | ✓ | ✓ (5 chapters) |
| File Verification | ✓ | ✓ |
| Manifest Validation | ✓ | ✓ |
| Storage Stats | ✓ | ✓ |
| Chapter Listing | ✓ | ✓ |
| Deletion | ✓ | ✓ (concurrent) |
| Race Conditions | - | ✓ |
| Corruption Detection | - | ✓ |
## Integration with CI/CD
These tests can be integrated into a CI/CD pipeline:
```yaml
# Example GitHub Actions workflow
- name: Start Server
run: node server.js &
- name: Wait for Server
run: sleep 5
- name: Run VPS Flow Tests
run: node test-vps-flow.js
- name: Run Concurrent Tests
run: node test-concurrent-downloads.js
```
## Performance Notes
- **VPS Flow Test**: ~2-3 minutes (downloads 5 images from 1 chapter)
- **Concurrent Test**: ~3-5 minutes (downloads 5 images from 5 chapters with max 3 concurrent)
Times vary based on:
- Network speed to source website
- VPS performance
- Current load on source website
## Contributing
When adding new features:
1. Add corresponding tests in `test-vps-flow.js`
2. If feature involves concurrent operations, add tests in `test-concurrent-downloads.js`
3. Update this README with new test coverage
4. Ensure all tests pass before submitting
## License
Same as the main MangaReader project.

316
backend/TEST_SUMMARY.md
View File

@@ -1,316 +0,0 @@
# Integration Tests: Creation Summary
## Overview
I have created comprehensive integration tests for the complete VPS flow: iOS app → VPS backend → Storage → Images served back.
## Files Created
### 1. `/home/ren/ios/MangaReader/backend/test-vps-flow.js`
**Purpose**: End-to-end integration test for the complete VPS download and serving flow
**Test Cases (11 tests)**:
- Server health check
- Get chapter images from scraper
- Download chapter to storage
- Verify chapter exists in storage
- Verify image files exist on disk
- Get image path from storage service
- List downloaded chapters
- Get storage statistics
- Delete chapter from storage
- Verify chapter was removed
- Verify storage stats updated after deletion
**Features**:
- Color-coded output for easy reading
- Detailed assertions with success/failure indicators
- Comprehensive error reporting
- Automatic cleanup
- Progress tracking
**Usage**:
```bash
npm run test:vps
# or
node test-vps-flow.js
```
### 2. `/home/ren/ios/MangaReader/backend/test-concurrent-downloads.js`
**Purpose**: Test concurrent download operations to verify thread safety and data integrity
**Test Cases (10 tests)**:
- Pre-download verification and cleanup
- Concurrent downloads (5 chapters, max 3 concurrent)
- Post-download verification
- Integrity check (no corruption, no missing files)
- Manifest independence verification
- Storage statistics accuracy
- Chapter listing functionality
- Concurrent deletion of all chapters
- Complete cleanup verification
- Race condition detection
**Features**:
- Progress tracker with operation IDs
- Batch processing (max 3 concurrent)
- Detailed integrity reports per chapter
- Corruption detection
- Missing file detection
- Concurrent operation tracking
- Race condition analysis
**Usage**:
```bash
npm run test:concurrent
# or
node test-concurrent-downloads.js
```
### 3. `/home/ren/ios/MangaReader/backend/run-tests.sh`
**Purpose**: Automation script for easy test execution and server management
**Commands**:
- `start` - Start server in background
- `stop` - Stop server
- `restart` - Restart server
- `logs` - Show server logs (tail -f)
- `status` - Check server status
- `vps-flow` - Run VPS flow test
- `concurrent` - Run concurrent downloads test
- `all` - Run all tests
- `cleanup` - Clean up test data
- `help` - Show help message
**Features**:
- Automatic server management
- PID tracking
- Log management
- Color-coded output
- Error handling
**Usage**:
```bash
./run-tests.sh start && ./run-tests.sh all && ./run-tests.sh cleanup && ./run-tests.sh stop
```
### 4. `/home/ren/ios/MangaReader/backend/TEST_README.md`
**Purpose**: Comprehensive documentation for integration tests
**Contents**:
- Detailed test descriptions
- Configuration options
- Prerequisites
- Usage examples
- Troubleshooting guide
- Test coverage table
- CI/CD integration examples
- Performance notes
### 5. `/home/ren/ios/MangaReader/backend/TEST_QUICK_START.md`
**Purpose**: Quick reference guide for running tests
**Contents**:
- Quick start instructions
- Multiple execution methods
- What gets tested
- Expected output
- Troubleshooting
- Test duration estimates
- Storage location info
### 6. Updated `/home/ren/ios/MangaReader/backend/package.json`
**Added npm scripts**:
- `test` - Run default tests
- `test:vps` - Run VPS flow test
- `test:concurrent` - Run concurrent downloads test
- `test:all` - Run all tests
- `test:clean` - Clean up test data
## Test Coverage Summary
| Feature | VPS Flow Test | Concurrent Test | Total Tests |
|---------|---------------|-----------------|-------------|
| Server Health | ✓ | - | 1 |
| Image Scraping | ✓ | ✓ | 2 |
| Download to Storage | ✓ | ✓ | 2 |
| File Verification | ✓ | ✓ | 2 |
| Manifest Validation | ✓ | ✓ | 2 |
| Storage Stats | ✓ | ✓ | 2 |
| Chapter Listing | ✓ | ✓ | 2 |
| Deletion | ✓ | ✓ | 2 |
| Cleanup | ✓ | ✓ | 2 |
| Race Conditions | - | ✓ | 1 |
| Corruption Detection | - | ✓ | 1 |
| **TOTAL** | **11** | **10** | **21** |
## Key Features Implemented
### 1. Comprehensive Logging
- Color-coded output (green for success, red for errors, blue for info)
- Detailed progress tracking
- Error messages with stack traces
- Operation tracking with IDs (for concurrent tests)
### 2. Robust Assertions
- Custom assertion functions with detailed messages
- Immediate feedback on failures
- Clear error context
### 3. Automatic Cleanup
- Tests clean up after themselves
- No residual test data
- Storage state restored
### 4. Progress Tracking
- Real-time operation status
- Duration tracking
- Batch processing information
- Summary statistics
### 5. Integrity Verification
- File existence checks
- Size validation
- Manifest validation
- Corruption detection
- Race condition detection
## Test Configuration
Both tests use these defaults (configurable in files):
```javascript
{
mangaSlug: 'one-piece_1695365223767',
chapters: [787, 788, 789, 790, 791], // Concurrent test only
baseUrl: 'http://localhost:3000',
timeout: 120000-180000 // 2-3 minutes
}
```
## Running the Tests
### Quick Start:
```bash
cd /home/ren/ios/MangaReader/backend
# Method 1: Using npm scripts
npm start # Terminal 1: Start server
npm run test:vps # Terminal 2: Run VPS flow test
npm run test:concurrent # Terminal 3: Run concurrent test
# Method 2: Using automation script
./run-tests.sh start
./run-tests.sh all
./run-tests.sh cleanup
./run-tests.sh stop
# Method 3: All in one
./run-tests.sh start && ./run-tests.sh all && ./run-tests.sh cleanup && ./run-tests.sh stop
```
## Expected Results
### Success Output:
```
============================================================
TEST RESULTS SUMMARY
============================================================
Total Tests: 11
Passed: 11
Failed: 0
======================================================================
🎉 ALL TESTS PASSED!
======================================================================
```
### Test Files Created During Execution:
```
/home/ren/ios/MangaReader/storage/manga/one-piece_1695365223767/
├── chapter_789/
│ ├── page_001.jpg
│ ├── page_002.jpg
│ ├── ...
│ └── manifest.json
```
## Assertions Included
Each test includes multiple assertions:
- **Equality checks** - Verify expected values match actual values
- **Truthy checks** - Verify conditions are met
- **File system checks** - Verify files and directories exist
- **Data validation** - Verify data integrity
- **Operation checks** - Verify operations complete successfully
## Error Handling
- Try-catch blocks around all operations
- Detailed error messages
- Stack traces for debugging
- Graceful failure handling
- Cleanup even on failure
## Performance Characteristics
- **VPS Flow Test**: Downloads 5 images (1 chapter) in ~2-3 minutes
- **Concurrent Test**: Downloads 25 images (5 chapters × 5 images) in ~3-5 minutes
- **Memory Usage**: Efficient concurrent processing with max 3 parallel downloads
- **Disk I/O**: Optimized for SSD/NVMe storage
## Next Steps
1. **Run the tests**:
```bash
cd /home/ren/ios/MangaReader/backend
./run-tests.sh all
```
2. **Verify results**: Check for green checkmarks and "ALL TESTS PASSED" message
3. **Review logs**: Check `logs/server.log` for any issues
4. **Inspect storage**: Verify downloaded images in storage directory
5. **Integrate into CI/CD**: Add to your CI/CD pipeline (see TEST_README.md)
## Maintenance
### Adding New Tests:
1. Create test function in appropriate test file
2. Add assertions using provided helper functions
3. Record test results
4. Update documentation
### Modifying Configuration:
- Edit `TEST_CONFIG` object in test files
- Update documentation if defaults change
### Extending Coverage:
- Add new test cases to existing suites
- Create new test files for new features
- Update TEST_README.md with coverage table
## Support
For issues or questions:
- Check TEST_README.md for detailed documentation
- Check TEST_QUICK_START.md for quick reference
- Review test output for specific error messages
- Check logs/server.log for server-side issues
## Summary
✅ Created 2 comprehensive test files with 21 total tests
✅ Created automation script for easy test execution
✅ Created detailed documentation (3 markdown files)
✅ Added npm scripts to package.json
✅ Implemented color-coded output and progress tracking
✅ Added comprehensive error handling and cleanup
✅ Verified thread safety and race condition detection
✅ Implemented integrity checks for file corruption
✅ Ready for CI/CD integration
All tests are production-ready and can be run immediately!