renato97/claude-config
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial: Claude config with agents, skills, commands, rules and scripts

Browse Source
This commit is contained in:
renato97
2026-02-16 20:21:30 -03:00
commit 8779f3a0a4
153 changed files with 27484 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
skills/android-reverse.md Normal file
View File

@@ -0,0 +1,61 @@
# Android Reverse Engineering Skill
## Installed Tools
- adb (Android Debug Bridge)
- fastboot
- apktool 2.11.1
- jadx 1.4.7 (CLI and GUI)
- apksigner
- zipalign
## Common Workflows
### APK Analysis
```bash
# Decompile APK
apktool d app.apk -o app_decompiled
# Recompile APK
apktool b app_decompiled -o app_modified.apk
# Sign APK
apksigner sign --ks keystore.jks app_modified.apk
# Analyze with JADX GUI
jadx-gui app.apk
# Analyze with JADX CLI
jadx -d output_dir app.apk
```
### ADB Commands
```bash
adb devices # List devices
adb shell # Open shell
adb install app.apk # Install APK
adb uninstall com.package.name # Uninstall
adb logcat # View logs
adb pull /sdcard/file.txt . # Download file
adb push file.txt /sdcard/ # Upload file
adb shell pm list packages # List installed packages
```
### APK Modification
```bash
# Full workflow
apktool d original.apk -o work_dir
# ... modify smali/resources ...
apktool b work_dir -o unsigned.apk
zipalign -v 4 unsigned.apk aligned.apk
apksigner sign --ks mykey.jks aligned.apk
```
## Tool Locations
- apktool: /opt/tools/apktool
- jadx: /opt/tools/jadx/bin/jadx
- jadx-gui: /opt/tools/jadx/bin/jadx-gui
## Security Notes
- Always verify APK signatures before installation
- Use isolated environment for unknown APKs
- Be careful with modified APKs - they can bypass security

522
skills/api-design/SKILL.md Normal file
View File

@@ -0,0 +1,522 @@
---
name: api-design
description: REST API design patterns including resource naming, status codes, pagination, filtering, error responses, versioning, and rate limiting for production APIs.
---
# API Design Patterns
Conventions and best practices for designing consistent, developer-friendly REST APIs.
## When to Activate
- Designing new API endpoints
- Reviewing existing API contracts
- Adding pagination, filtering, or sorting
- Implementing error handling for APIs
- Planning API versioning strategy
- Building public or partner-facing APIs
## Resource Design
### URL Structure
```
# Resources are nouns, plural, lowercase, kebab-case
GET /api/v1/users
GET /api/v1/users/:id
POST /api/v1/users
PUT /api/v1/users/:id
PATCH /api/v1/users/:id
DELETE /api/v1/users/:id
# Sub-resources for relationships
GET /api/v1/users/:id/orders
POST /api/v1/users/:id/orders
# Actions that don't map to CRUD (use verbs sparingly)
POST /api/v1/orders/:id/cancel
POST /api/v1/auth/login
POST /api/v1/auth/refresh
```
### Naming Rules
```
# GOOD
/api/v1/team-members # kebab-case for multi-word resources
/api/v1/orders?status=active # query params for filtering
/api/v1/users/123/orders # nested resources for ownership
# BAD
/api/v1/getUsers # verb in URL
/api/v1/user # singular (use plural)
/api/v1/team_members # snake_case in URLs
/api/v1/users/123/getOrders # verb in nested resource
```
## HTTP Methods and Status Codes
### Method Semantics
| Method | Idempotent | Safe | Use For |
|--------|-----------|------|---------|
| GET | Yes | Yes | Retrieve resources |
| POST | No | No | Create resources, trigger actions |
| PUT | Yes | No | Full replacement of a resource |
| PATCH | No* | No | Partial update of a resource |
| DELETE | Yes | No | Remove a resource |
*PATCH can be made idempotent with proper implementation
### Status Code Reference
```
# Success
200 OK — GET, PUT, PATCH (with response body)
201 Created — POST (include Location header)
204 No Content — DELETE, PUT (no response body)
# Client Errors
400 Bad Request — Validation failure, malformed JSON
401 Unauthorized — Missing or invalid authentication
403 Forbidden — Authenticated but not authorized
404 Not Found — Resource doesn't exist
409 Conflict — Duplicate entry, state conflict
422 Unprocessable Entity — Semantically invalid (valid JSON, bad data)
429 Too Many Requests — Rate limit exceeded
# Server Errors
500 Internal Server Error — Unexpected failure (never expose details)
502 Bad Gateway — Upstream service failed
503 Service Unavailable — Temporary overload, include Retry-After
```
### Common Mistakes
```
# BAD: 200 for everything
{ "status": 200, "success": false, "error": "Not found" }
# GOOD: Use HTTP status codes semantically
HTTP/1.1 404 Not Found
{ "error": { "code": "not_found", "message": "User not found" } }
# BAD: 500 for validation errors
# GOOD: 400 or 422 with field-level details
# BAD: 200 for created resources
# GOOD: 201 with Location header
HTTP/1.1 201 Created
Location: /api/v1/users/abc-123
```
## Response Format
### Success Response
```json
{
"data": {
"id": "abc-123",
"email": "alice@example.com",
"name": "Alice",
"created_at": "2025-01-15T10:30:00Z"
}
}
```
### Collection Response (with Pagination)
```json
{
"data": [
{ "id": "abc-123", "name": "Alice" },
{ "id": "def-456", "name": "Bob" }
],
"meta": {
"total": 142,
"page": 1,
"per_page": 20,
"total_pages": 8
},
"links": {
"self": "/api/v1/users?page=1&per_page=20",
"next": "/api/v1/users?page=2&per_page=20",
"last": "/api/v1/users?page=8&per_page=20"
}
}
```
### Error Response
```json
{
"error": {
"code": "validation_error",
"message": "Request validation failed",
"details": [
{
"field": "email",
"message": "Must be a valid email address",
"code": "invalid_format"
},
{
"field": "age",
"message": "Must be between 0 and 150",
"code": "out_of_range"
}
]
}
}
```
### Response Envelope Variants
```typescript
// Option A: Envelope with data wrapper (recommended for public APIs)
interface ApiResponse<T> {
data: T;
meta?: PaginationMeta;
links?: PaginationLinks;
}
interface ApiError {
error: {
code: string;
message: string;
details?: FieldError[];
};
}
// Option B: Flat response (simpler, common for internal APIs)
// Success: just return the resource directly
// Error: return error object
// Distinguish by HTTP status code
```
## Pagination
### Offset-Based (Simple)
```
GET /api/v1/users?page=2&per_page=20
# Implementation
SELECT * FROM users
ORDER BY created_at DESC
LIMIT 20 OFFSET 20;
```
**Pros:** Easy to implement, supports "jump to page N"
**Cons:** Slow on large offsets (OFFSET 100000), inconsistent with concurrent inserts
### Cursor-Based (Scalable)
```
GET /api/v1/users?cursor=eyJpZCI6MTIzfQ&limit=20
# Implementation
SELECT * FROM users
WHERE id > :cursor_id
ORDER BY id ASC
LIMIT 21; -- fetch one extra to determine has_next
```
```json
{
"data": [...],
"meta": {
"has_next": true,
"next_cursor": "eyJpZCI6MTQzfQ"
}
}
```
**Pros:** Consistent performance regardless of position, stable with concurrent inserts
**Cons:** Cannot jump to arbitrary page, cursor is opaque
### When to Use Which
| Use Case | Pagination Type |
|----------|----------------|
| Admin dashboards, small datasets (<10K) | Offset |
| Infinite scroll, feeds, large datasets | Cursor |
| Public APIs | Cursor (default) with offset (optional) |
| Search results | Offset (users expect page numbers) |
## Filtering, Sorting, and Search
### Filtering
```
# Simple equality
GET /api/v1/orders?status=active&customer_id=abc-123
# Comparison operators (use bracket notation)
GET /api/v1/products?price[gte]=10&price[lte]=100
GET /api/v1/orders?created_at[after]=2025-01-01
# Multiple values (comma-separated)
GET /api/v1/products?category=electronics,clothing
# Nested fields (dot notation)
GET /api/v1/orders?customer.country=US
```
### Sorting
```
# Single field (prefix - for descending)
GET /api/v1/products?sort=-created_at
# Multiple fields (comma-separated)
GET /api/v1/products?sort=-featured,price,-created_at
```
### Full-Text Search
```
# Search query parameter
GET /api/v1/products?q=wireless+headphones
# Field-specific search
GET /api/v1/users?email=alice
```
### Sparse Fieldsets
```
# Return only specified fields (reduces payload)
GET /api/v1/users?fields=id,name,email
GET /api/v1/orders?fields=id,total,status&include=customer.name
```
## Authentication and Authorization
### Token-Based Auth
```
# Bearer token in Authorization header
GET /api/v1/users
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...
# API key (for server-to-server)
GET /api/v1/data
X-API-Key: sk_live_abc123
```
### Authorization Patterns
```typescript
// Resource-level: check ownership
app.get("/api/v1/orders/:id", async (req, res) => {
const order = await Order.findById(req.params.id);
if (!order) return res.status(404).json({ error: { code: "not_found" } });
if (order.userId !== req.user.id) return res.status(403).json({ error: { code: "forbidden" } });
return res.json({ data: order });
});
// Role-based: check permissions
app.delete("/api/v1/users/:id", requireRole("admin"), async (req, res) => {
await User.delete(req.params.id);
return res.status(204).send();
});
```
## Rate Limiting
### Headers
```
HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1640000000
# When exceeded
HTTP/1.1 429 Too Many Requests
Retry-After: 60
{
"error": {
"code": "rate_limit_exceeded",
"message": "Rate limit exceeded. Try again in 60 seconds."
}
}
```
### Rate Limit Tiers
| Tier | Limit | Window | Use Case |
|------|-------|--------|----------|
| Anonymous | 30/min | Per IP | Public endpoints |
| Authenticated | 100/min | Per user | Standard API access |
| Premium | 1000/min | Per API key | Paid API plans |
| Internal | 10000/min | Per service | Service-to-service |
## Versioning
### URL Path Versioning (Recommended)
```
/api/v1/users
/api/v2/users
```
**Pros:** Explicit, easy to route, cacheable
**Cons:** URL changes between versions
### Header Versioning
```
GET /api/users
Accept: application/vnd.myapp.v2+json
```
**Pros:** Clean URLs
**Cons:** Harder to test, easy to forget
### Versioning Strategy
```
1. Start with /api/v1/ — don't version until you need to
2. Maintain at most 2 active versions (current + previous)
3. Deprecation timeline:
- Announce deprecation (6 months notice for public APIs)
- Add Sunset header: Sunset: Sat, 01 Jan 2026 00:00:00 GMT
- Return 410 Gone after sunset date
4. Non-breaking changes don't need a new version:
- Adding new fields to responses
- Adding new optional query parameters
- Adding new endpoints
5. Breaking changes require a new version:
- Removing or renaming fields
- Changing field types
- Changing URL structure
- Changing authentication method
```
## Implementation Patterns
### TypeScript (Next.js API Route)
```typescript
import { z } from "zod";
import { NextRequest, NextResponse } from "next/server";
const createUserSchema = z.object({
email: z.string().email(),
name: z.string().min(1).max(100),
});
export async function POST(req: NextRequest) {
const body = await req.json();
const parsed = createUserSchema.safeParse(body);
if (!parsed.success) {
return NextResponse.json({
error: {
code: "validation_error",
message: "Request validation failed",
details: parsed.error.issues.map(i => ({
field: i.path.join("."),
message: i.message,
code: i.code,
})),
},
}, { status: 422 });
}
const user = await createUser(parsed.data);
return NextResponse.json(
{ data: user },
{
status: 201,
headers: { Location: `/api/v1/users/${user.id}` },
},
);
}
```
### Python (Django REST Framework)
```python
from rest_framework import serializers, viewsets, status
from rest_framework.response import Response
class CreateUserSerializer(serializers.Serializer):
email = serializers.EmailField()
name = serializers.CharField(max_length=100)
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = ["id", "email", "name", "created_at"]
class UserViewSet(viewsets.ModelViewSet):
serializer_class = UserSerializer
permission_classes = [IsAuthenticated]
def get_serializer_class(self):
if self.action == "create":
return CreateUserSerializer
return UserSerializer
def create(self, request):
serializer = CreateUserSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
user = UserService.create(**serializer.validated_data)
return Response(
{"data": UserSerializer(user).data},
status=status.HTTP_201_CREATED,
headers={"Location": f"/api/v1/users/{user.id}"},
)
```
### Go (net/http)
```go
func (h *UserHandler) CreateUser(w http.ResponseWriter, r *http.Request) {
var req CreateUserRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", "Invalid request body")
return
}
if err := req.Validate(); err != nil {
writeError(w, http.StatusUnprocessableEntity, "validation_error", err.Error())
return
}
user, err := h.service.Create(r.Context(), req)
if err != nil {
switch {
case errors.Is(err, domain.ErrEmailTaken):
writeError(w, http.StatusConflict, "email_taken", "Email already registered")
default:
writeError(w, http.StatusInternalServerError, "internal_error", "Internal error")
}
return
}
w.Header().Set("Location", fmt.Sprintf("/api/v1/users/%s", user.ID))
writeJSON(w, http.StatusCreated, map[string]any{"data": user})
}
```
## API Design Checklist
Before shipping a new endpoint:
- [ ] Resource URL follows naming conventions (plural, kebab-case, no verbs)
- [ ] Correct HTTP method used (GET for reads, POST for creates, etc.)
- [ ] Appropriate status codes returned (not 200 for everything)
- [ ] Input validated with schema (Zod, Pydantic, Bean Validation)
- [ ] Error responses follow standard format with codes and messages
- [ ] Pagination implemented for list endpoints (cursor or offset)
- [ ] Authentication required (or explicitly marked as public)
- [ ] Authorization checked (user can only access their own resources)
- [ ] Rate limiting configured
- [ ] Response does not leak internal details (stack traces, SQL errors)
- [ ] Consistent naming with existing endpoints (camelCase vs snake_case)
- [ ] Documented (OpenAPI/Swagger spec updated)

597
skills/backend-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,597 @@
---
name: backend-patterns
description: Backend architecture patterns, API design, database optimization, and server-side best practices for Node.js, Express, and Next.js API routes.
---
# Backend Development Patterns
Backend architecture patterns and best practices for scalable server-side applications.
## When to Activate
- Designing REST or GraphQL API endpoints
- Implementing repository, service, or controller layers
- Optimizing database queries (N+1, indexing, connection pooling)
- Adding caching (Redis, in-memory, HTTP cache headers)
- Setting up background jobs or async processing
- Structuring error handling and validation for APIs
- Building middleware (auth, logging, rate limiting)
## API Design Patterns
### RESTful API Structure
```typescript
// ✅ Resource-based URLs
GET /api/markets # List resources
GET /api/markets/:id # Get single resource
POST /api/markets # Create resource
PUT /api/markets/:id # Replace resource
PATCH /api/markets/:id # Update resource
DELETE /api/markets/:id # Delete resource
// ✅ Query parameters for filtering, sorting, pagination
GET /api/markets?status=active&sort=volume&limit=20&offset=0
```
### Repository Pattern
```typescript
// Abstract data access logic
interface MarketRepository {
findAll(filters?: MarketFilters): Promise<Market[]>
findById(id: string): Promise<Market | null>
create(data: CreateMarketDto): Promise<Market>
update(id: string, data: UpdateMarketDto): Promise<Market>
delete(id: string): Promise<void>
}
class SupabaseMarketRepository implements MarketRepository {
async findAll(filters?: MarketFilters): Promise<Market[]> {
let query = supabase.from('markets').select('*')
if (filters?.status) {
query = query.eq('status', filters.status)
}
if (filters?.limit) {
query = query.limit(filters.limit)
}
const { data, error } = await query
if (error) throw new Error(error.message)
return data
}
// Other methods...
}
```
### Service Layer Pattern
```typescript
// Business logic separated from data access
class MarketService {
constructor(private marketRepo: MarketRepository) {}
async searchMarkets(query: string, limit: number = 10): Promise<Market[]> {
// Business logic
const embedding = await generateEmbedding(query)
const results = await this.vectorSearch(embedding, limit)
// Fetch full data
const markets = await this.marketRepo.findByIds(results.map(r => r.id))
// Sort by similarity
return markets.sort((a, b) => {
const scoreA = results.find(r => r.id === a.id)?.score || 0
const scoreB = results.find(r => r.id === b.id)?.score || 0
return scoreA - scoreB
})
}
private async vectorSearch(embedding: number[], limit: number) {
// Vector search implementation
}
}
```
### Middleware Pattern
```typescript
// Request/response processing pipeline
export function withAuth(handler: NextApiHandler): NextApiHandler {
return async (req, res) => {
const token = req.headers.authorization?.replace('Bearer ', '')
if (!token) {
return res.status(401).json({ error: 'Unauthorized' })
}
try {
const user = await verifyToken(token)
req.user = user
return handler(req, res)
} catch (error) {
return res.status(401).json({ error: 'Invalid token' })
}
}
}
// Usage
export default withAuth(async (req, res) => {
// Handler has access to req.user
})
```
## Database Patterns
### Query Optimization
```typescript
// ✅ GOOD: Select only needed columns
const { data } = await supabase
.from('markets')
.select('id, name, status, volume')
.eq('status', 'active')
.order('volume', { ascending: false })
.limit(10)
// ❌ BAD: Select everything
const { data } = await supabase
.from('markets')
.select('*')
```
### N+1 Query Prevention
```typescript
// ❌ BAD: N+1 query problem
const markets = await getMarkets()
for (const market of markets) {
market.creator = await getUser(market.creator_id) // N queries
}
// ✅ GOOD: Batch fetch
const markets = await getMarkets()
const creatorIds = markets.map(m => m.creator_id)
const creators = await getUsers(creatorIds) // 1 query
const creatorMap = new Map(creators.map(c => [c.id, c]))
markets.forEach(market => {
market.creator = creatorMap.get(market.creator_id)
})
```
### Transaction Pattern
```typescript
async function createMarketWithPosition(
marketData: CreateMarketDto,
positionData: CreatePositionDto
) {
// Use Supabase transaction
const { data, error } = await supabase.rpc('create_market_with_position', {
market_data: marketData,
position_data: positionData
})
if (error) throw new Error('Transaction failed')
return data
}
// SQL function in Supabase
CREATE OR REPLACE FUNCTION create_market_with_position(
market_data jsonb,
position_data jsonb
)
RETURNS jsonb
LANGUAGE plpgsql
AS $$
BEGIN
-- Start transaction automatically
INSERT INTO markets VALUES (market_data);
INSERT INTO positions VALUES (position_data);
RETURN jsonb_build_object('success', true);
EXCEPTION
WHEN OTHERS THEN
-- Rollback happens automatically
RETURN jsonb_build_object('success', false, 'error', SQLERRM);
END;
$$;
```
## Caching Strategies
### Redis Caching Layer
```typescript
class CachedMarketRepository implements MarketRepository {
constructor(
private baseRepo: MarketRepository,
private redis: RedisClient
) {}
async findById(id: string): Promise<Market | null> {
// Check cache first
const cached = await this.redis.get(`market:${id}`)
if (cached) {
return JSON.parse(cached)
}
// Cache miss - fetch from database
const market = await this.baseRepo.findById(id)
if (market) {
// Cache for 5 minutes
await this.redis.setex(`market:${id}`, 300, JSON.stringify(market))
}
return market
}
async invalidateCache(id: string): Promise<void> {
await this.redis.del(`market:${id}`)
}
}
```
### Cache-Aside Pattern
```typescript
async function getMarketWithCache(id: string): Promise<Market> {
const cacheKey = `market:${id}`
// Try cache
const cached = await redis.get(cacheKey)
if (cached) return JSON.parse(cached)
// Cache miss - fetch from DB
const market = await db.markets.findUnique({ where: { id } })
if (!market) throw new Error('Market not found')
// Update cache
await redis.setex(cacheKey, 300, JSON.stringify(market))
return market
}
```
## Error Handling Patterns
### Centralized Error Handler
```typescript
class ApiError extends Error {
constructor(
public statusCode: number,
public message: string,
public isOperational = true
) {
super(message)
Object.setPrototypeOf(this, ApiError.prototype)
}
}
export function errorHandler(error: unknown, req: Request): Response {
if (error instanceof ApiError) {
return NextResponse.json({
success: false,
error: error.message
}, { status: error.statusCode })
}
if (error instanceof z.ZodError) {
return NextResponse.json({
success: false,
error: 'Validation failed',
details: error.errors
}, { status: 400 })
}
// Log unexpected errors
console.error('Unexpected error:', error)
return NextResponse.json({
success: false,
error: 'Internal server error'
}, { status: 500 })
}
// Usage
export async function GET(request: Request) {
try {
const data = await fetchData()
return NextResponse.json({ success: true, data })
} catch (error) {
return errorHandler(error, request)
}
}
```
### Retry with Exponential Backoff
```typescript
async function fetchWithRetry<T>(
fn: () => Promise<T>,
maxRetries = 3
): Promise<T> {
let lastError: Error
for (let i = 0; i < maxRetries; i++) {
try {
return await fn()
} catch (error) {
lastError = error as Error
if (i < maxRetries - 1) {
// Exponential backoff: 1s, 2s, 4s
const delay = Math.pow(2, i) * 1000
await new Promise(resolve => setTimeout(resolve, delay))
}
}
}
throw lastError!
}
// Usage
const data = await fetchWithRetry(() => fetchFromAPI())
```
## Authentication & Authorization
### JWT Token Validation
```typescript
import jwt from 'jsonwebtoken'
interface JWTPayload {
userId: string
email: string
role: 'admin' | 'user'
}
export function verifyToken(token: string): JWTPayload {
try {
const payload = jwt.verify(token, process.env.JWT_SECRET!) as JWTPayload
return payload
} catch (error) {
throw new ApiError(401, 'Invalid token')
}
}
export async function requireAuth(request: Request) {
const token = request.headers.get('authorization')?.replace('Bearer ', '')
if (!token) {
throw new ApiError(401, 'Missing authorization token')
}
return verifyToken(token)
}
// Usage in API route
export async function GET(request: Request) {
const user = await requireAuth(request)
const data = await getDataForUser(user.userId)
return NextResponse.json({ success: true, data })
}
```
### Role-Based Access Control
```typescript
type Permission = 'read' | 'write' | 'delete' | 'admin'
interface User {
id: string
role: 'admin' | 'moderator' | 'user'
}
const rolePermissions: Record<User['role'], Permission[]> = {
admin: ['read', 'write', 'delete', 'admin'],
moderator: ['read', 'write', 'delete'],
user: ['read', 'write']
}
export function hasPermission(user: User, permission: Permission): boolean {
return rolePermissions[user.role].includes(permission)
}
export function requirePermission(permission: Permission) {
return (handler: (request: Request, user: User) => Promise<Response>) => {
return async (request: Request) => {
const user = await requireAuth(request)
if (!hasPermission(user, permission)) {
throw new ApiError(403, 'Insufficient permissions')
}
return handler(request, user)
}
}
}
// Usage - HOF wraps the handler
export const DELETE = requirePermission('delete')(
async (request: Request, user: User) => {
// Handler receives authenticated user with verified permission
return new Response('Deleted', { status: 200 })
}
)
```
## Rate Limiting
### Simple In-Memory Rate Limiter
```typescript
class RateLimiter {
private requests = new Map<string, number[]>()
async checkLimit(
identifier: string,
maxRequests: number,
windowMs: number
): Promise<boolean> {
const now = Date.now()
const requests = this.requests.get(identifier) || []
// Remove old requests outside window
const recentRequests = requests.filter(time => now - time < windowMs)
if (recentRequests.length >= maxRequests) {
return false // Rate limit exceeded
}
// Add current request
recentRequests.push(now)
this.requests.set(identifier, recentRequests)
return true
}
}
const limiter = new RateLimiter()
export async function GET(request: Request) {
const ip = request.headers.get('x-forwarded-for') || 'unknown'
const allowed = await limiter.checkLimit(ip, 100, 60000) // 100 req/min
if (!allowed) {
return NextResponse.json({
error: 'Rate limit exceeded'
}, { status: 429 })
}
// Continue with request
}
```
## Background Jobs & Queues
### Simple Queue Pattern
```typescript
class JobQueue<T> {
private queue: T[] = []
private processing = false
async add(job: T): Promise<void> {
this.queue.push(job)
if (!this.processing) {
this.process()
}
}
private async process(): Promise<void> {
this.processing = true
while (this.queue.length > 0) {
const job = this.queue.shift()!
try {
await this.execute(job)
} catch (error) {
console.error('Job failed:', error)
}
}
this.processing = false
}
private async execute(job: T): Promise<void> {
// Job execution logic
}
}
// Usage for indexing markets
interface IndexJob {
marketId: string
}
const indexQueue = new JobQueue<IndexJob>()
export async function POST(request: Request) {
const { marketId } = await request.json()
// Add to queue instead of blocking
await indexQueue.add({ marketId })
return NextResponse.json({ success: true, message: 'Job queued' })
}
```
## Logging & Monitoring
### Structured Logging
```typescript
interface LogContext {
userId?: string
requestId?: string
method?: string
path?: string
[key: string]: unknown
}
class Logger {
log(level: 'info' | 'warn' | 'error', message: string, context?: LogContext) {
const entry = {
timestamp: new Date().toISOString(),
level,
message,
...context
}
console.log(JSON.stringify(entry))
}
info(message: string, context?: LogContext) {
this.log('info', message, context)
}
warn(message: string, context?: LogContext) {
this.log('warn', message, context)
}
error(message: string, error: Error, context?: LogContext) {
this.log('error', message, {
...context,
error: error.message,
stack: error.stack
})
}
}
const logger = new Logger()
// Usage
export async function GET(request: Request) {
const requestId = crypto.randomUUID()
logger.info('Fetching markets', {
requestId,
method: 'GET',
path: '/api/markets'
})
try {
const markets = await fetchMarkets()
return NextResponse.json({ success: true, data: markets })
} catch (error) {
logger.error('Failed to fetch markets', error as Error, { requestId })
return NextResponse.json({ error: 'Internal error' }, { status: 500 })
}
}
```
**Remember**: Backend patterns enable scalable, maintainable server-side applications. Choose patterns that fit your complexity level.

438
skills/clickhouse-io/SKILL.md Normal file
View File

@@ -0,0 +1,438 @@
---
name: clickhouse-io
description: ClickHouse database patterns, query optimization, analytics, and data engineering best practices for high-performance analytical workloads.
---
# ClickHouse Analytics Patterns
ClickHouse-specific patterns for high-performance analytics and data engineering.
## When to Activate
- Designing ClickHouse table schemas (MergeTree engine selection)
- Writing analytical queries (aggregations, window functions, joins)
- Optimizing query performance (partition pruning, projections, materialized views)
- Ingesting large volumes of data (batch inserts, Kafka integration)
- Migrating from PostgreSQL/MySQL to ClickHouse for analytics
- Implementing real-time dashboards or time-series analytics
## Overview
ClickHouse is a column-oriented database management system (DBMS) for online analytical processing (OLAP). It's optimized for fast analytical queries on large datasets.
**Key Features:**
- Column-oriented storage
- Data compression
- Parallel query execution
- Distributed queries
- Real-time analytics
## Table Design Patterns
### MergeTree Engine (Most Common)
```sql
CREATE TABLE markets_analytics (
date Date,
market_id String,
market_name String,
volume UInt64,
trades UInt32,
unique_traders UInt32,
avg_trade_size Float64,
created_at DateTime
) ENGINE = MergeTree()
PARTITION BY toYYYYMM(date)
ORDER BY (date, market_id)
SETTINGS index_granularity = 8192;
```
### ReplacingMergeTree (Deduplication)
```sql
-- For data that may have duplicates (e.g., from multiple sources)
CREATE TABLE user_events (
event_id String,
user_id String,
event_type String,
timestamp DateTime,
properties String
) ENGINE = ReplacingMergeTree()
PARTITION BY toYYYYMM(timestamp)
ORDER BY (user_id, event_id, timestamp)
PRIMARY KEY (user_id, event_id);
```
### AggregatingMergeTree (Pre-aggregation)
```sql
-- For maintaining aggregated metrics
CREATE TABLE market_stats_hourly (
hour DateTime,
market_id String,
total_volume AggregateFunction(sum, UInt64),
total_trades AggregateFunction(count, UInt32),
unique_users AggregateFunction(uniq, String)
) ENGINE = AggregatingMergeTree()
PARTITION BY toYYYYMM(hour)
ORDER BY (hour, market_id);
-- Query aggregated data
SELECT
hour,
market_id,
sumMerge(total_volume) AS volume,
countMerge(total_trades) AS trades,
uniqMerge(unique_users) AS users
FROM market_stats_hourly
WHERE hour >= toStartOfHour(now() - INTERVAL 24 HOUR)
GROUP BY hour, market_id
ORDER BY hour DESC;
```
## Query Optimization Patterns
### Efficient Filtering
```sql
-- ✅ GOOD: Use indexed columns first
SELECT *
FROM markets_analytics
WHERE date >= '2025-01-01'
AND market_id = 'market-123'
AND volume > 1000
ORDER BY date DESC
LIMIT 100;
-- ❌ BAD: Filter on non-indexed columns first
SELECT *
FROM markets_analytics
WHERE volume > 1000
AND market_name LIKE '%election%'
AND date >= '2025-01-01';
```
### Aggregations
```sql
-- ✅ GOOD: Use ClickHouse-specific aggregation functions
SELECT
toStartOfDay(created_at) AS day,
market_id,
sum(volume) AS total_volume,
count() AS total_trades,
uniq(trader_id) AS unique_traders,
avg(trade_size) AS avg_size
FROM trades
WHERE created_at >= today() - INTERVAL 7 DAY
GROUP BY day, market_id
ORDER BY day DESC, total_volume DESC;
-- ✅ Use quantile for percentiles (more efficient than percentile)
SELECT
quantile(0.50)(trade_size) AS median,
quantile(0.95)(trade_size) AS p95,
quantile(0.99)(trade_size) AS p99
FROM trades
WHERE created_at >= now() - INTERVAL 1 HOUR;
```
### Window Functions
```sql
-- Calculate running totals
SELECT
date,
market_id,
volume,
sum(volume) OVER (
PARTITION BY market_id
ORDER BY date
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS cumulative_volume
FROM markets_analytics
WHERE date >= today() - INTERVAL 30 DAY
ORDER BY market_id, date;
```
## Data Insertion Patterns
### Bulk Insert (Recommended)
```typescript
import { ClickHouse } from 'clickhouse'
const clickhouse = new ClickHouse({
url: process.env.CLICKHOUSE_URL,
port: 8123,
basicAuth: {
username: process.env.CLICKHOUSE_USER,
password: process.env.CLICKHOUSE_PASSWORD
}
})
// ✅ Batch insert (efficient)
async function bulkInsertTrades(trades: Trade[]) {
const values = trades.map(trade => `(
'${trade.id}',
'${trade.market_id}',
'${trade.user_id}',
${trade.amount},
'${trade.timestamp.toISOString()}'
)`).join(',')
await clickhouse.query(`
INSERT INTO trades (id, market_id, user_id, amount, timestamp)
VALUES ${values}
`).toPromise()
}
// ❌ Individual inserts (slow)
async function insertTrade(trade: Trade) {
// Don't do this in a loop!
await clickhouse.query(`
INSERT INTO trades VALUES ('${trade.id}', ...)
`).toPromise()
}
```
### Streaming Insert
```typescript
// For continuous data ingestion
import { createWriteStream } from 'fs'
import { pipeline } from 'stream/promises'
async function streamInserts() {
const stream = clickhouse.insert('trades').stream()
for await (const batch of dataSource) {
stream.write(batch)
}
await stream.end()
}
```
## Materialized Views
### Real-time Aggregations
```sql
-- Create materialized view for hourly stats
CREATE MATERIALIZED VIEW market_stats_hourly_mv
TO market_stats_hourly
AS SELECT
toStartOfHour(timestamp) AS hour,
market_id,
sumState(amount) AS total_volume,
countState() AS total_trades,
uniqState(user_id) AS unique_users
FROM trades
GROUP BY hour, market_id;
-- Query the materialized view
SELECT
hour,
market_id,
sumMerge(total_volume) AS volume,
countMerge(total_trades) AS trades,
uniqMerge(unique_users) AS users
FROM market_stats_hourly
WHERE hour >= now() - INTERVAL 24 HOUR
GROUP BY hour, market_id;
```
## Performance Monitoring
### Query Performance
```sql
-- Check slow queries
SELECT
query_id,
user,
query,
query_duration_ms,
read_rows,
read_bytes,
memory_usage
FROM system.query_log
WHERE type = 'QueryFinish'
AND query_duration_ms > 1000
AND event_time >= now() - INTERVAL 1 HOUR
ORDER BY query_duration_ms DESC
LIMIT 10;
```
### Table Statistics
```sql
-- Check table sizes
SELECT
database,
table,
formatReadableSize(sum(bytes)) AS size,
sum(rows) AS rows,
max(modification_time) AS latest_modification
FROM system.parts
WHERE active
GROUP BY database, table
ORDER BY sum(bytes) DESC;
```
## Common Analytics Queries
### Time Series Analysis
```sql
-- Daily active users
SELECT
toDate(timestamp) AS date,
uniq(user_id) AS daily_active_users
FROM events
WHERE timestamp >= today() - INTERVAL 30 DAY
GROUP BY date
ORDER BY date;
-- Retention analysis
SELECT
signup_date,
countIf(days_since_signup = 0) AS day_0,
countIf(days_since_signup = 1) AS day_1,
countIf(days_since_signup = 7) AS day_7,
countIf(days_since_signup = 30) AS day_30
FROM (
SELECT
user_id,
min(toDate(timestamp)) AS signup_date,
toDate(timestamp) AS activity_date,
dateDiff('day', signup_date, activity_date) AS days_since_signup
FROM events
GROUP BY user_id, activity_date
)
GROUP BY signup_date
ORDER BY signup_date DESC;
```
### Funnel Analysis
```sql
-- Conversion funnel
SELECT
countIf(step = 'viewed_market') AS viewed,
countIf(step = 'clicked_trade') AS clicked,
countIf(step = 'completed_trade') AS completed,
round(clicked / viewed * 100, 2) AS view_to_click_rate,
round(completed / clicked * 100, 2) AS click_to_completion_rate
FROM (
SELECT
user_id,
session_id,
event_type AS step
FROM events
WHERE event_date = today()
)
GROUP BY session_id;
```
### Cohort Analysis
```sql
-- User cohorts by signup month
SELECT
toStartOfMonth(signup_date) AS cohort,
toStartOfMonth(activity_date) AS month,
dateDiff('month', cohort, month) AS months_since_signup,
count(DISTINCT user_id) AS active_users
FROM (
SELECT
user_id,
min(toDate(timestamp)) OVER (PARTITION BY user_id) AS signup_date,
toDate(timestamp) AS activity_date
FROM events
)
GROUP BY cohort, month, months_since_signup
ORDER BY cohort, months_since_signup;
```
## Data Pipeline Patterns
### ETL Pattern
```typescript
// Extract, Transform, Load
async function etlPipeline() {
// 1. Extract from source
const rawData = await extractFromPostgres()
// 2. Transform
const transformed = rawData.map(row => ({
date: new Date(row.created_at).toISOString().split('T')[0],
market_id: row.market_slug,
volume: parseFloat(row.total_volume),
trades: parseInt(row.trade_count)
}))
// 3. Load to ClickHouse
await bulkInsertToClickHouse(transformed)
}
// Run periodically
setInterval(etlPipeline, 60 * 60 * 1000) // Every hour
```
### Change Data Capture (CDC)
```typescript
// Listen to PostgreSQL changes and sync to ClickHouse
import { Client } from 'pg'
const pgClient = new Client({ connectionString: process.env.DATABASE_URL })
pgClient.query('LISTEN market_updates')
pgClient.on('notification', async (msg) => {
const update = JSON.parse(msg.payload)
await clickhouse.insert('market_updates', [
{
market_id: update.id,
event_type: update.operation, // INSERT, UPDATE, DELETE
timestamp: new Date(),
data: JSON.stringify(update.new_data)
}
])
})
```
## Best Practices
### 1. Partitioning Strategy
- Partition by time (usually month or day)
- Avoid too many partitions (performance impact)
- Use DATE type for partition key
### 2. Ordering Key
- Put most frequently filtered columns first
- Consider cardinality (high cardinality first)
- Order impacts compression
### 3. Data Types
- Use smallest appropriate type (UInt32 vs UInt64)
- Use LowCardinality for repeated strings
- Use Enum for categorical data
### 4. Avoid
- SELECT * (specify columns)
- FINAL (merge data before query instead)
- Too many JOINs (denormalize for analytics)
- Small frequent inserts (batch instead)
### 5. Monitoring
- Track query performance
- Monitor disk usage
- Check merge operations
- Review slow query log
**Remember**: ClickHouse excels at analytical workloads. Design tables for your query patterns, batch inserts, and leverage materialized views for real-time aggregations.

529
skills/coding-standards/SKILL.md Normal file
View File

@@ -0,0 +1,529 @@
---
name: coding-standards
description: Universal coding standards, best practices, and patterns for TypeScript, JavaScript, React, and Node.js development.
---
# Coding Standards & Best Practices
Universal coding standards applicable across all projects.
## When to Activate
- Starting a new project or module
- Reviewing code for quality and maintainability
- Refactoring existing code to follow conventions
- Enforcing naming, formatting, or structural consistency
- Setting up linting, formatting, or type-checking rules
- Onboarding new contributors to coding conventions
## Code Quality Principles
### 1. Readability First
- Code is read more than written
- Clear variable and function names
- Self-documenting code preferred over comments
- Consistent formatting
### 2. KISS (Keep It Simple, Stupid)
- Simplest solution that works
- Avoid over-engineering
- No premature optimization
- Easy to understand > clever code
### 3. DRY (Don't Repeat Yourself)
- Extract common logic into functions
- Create reusable components
- Share utilities across modules
- Avoid copy-paste programming
### 4. YAGNI (You Aren't Gonna Need It)
- Don't build features before they're needed
- Avoid speculative generality
- Add complexity only when required
- Start simple, refactor when needed
## TypeScript/JavaScript Standards
### Variable Naming
```typescript
// ✅ GOOD: Descriptive names
const marketSearchQuery = 'election'
const isUserAuthenticated = true
const totalRevenue = 1000
// ❌ BAD: Unclear names
const q = 'election'
const flag = true
const x = 1000
```
### Function Naming
```typescript
// ✅ GOOD: Verb-noun pattern
async function fetchMarketData(marketId: string) { }
function calculateSimilarity(a: number[], b: number[]) { }
function isValidEmail(email: string): boolean { }
// ❌ BAD: Unclear or noun-only
async function market(id: string) { }
function similarity(a, b) { }
function email(e) { }
```
### Immutability Pattern (CRITICAL)
```typescript
// ✅ ALWAYS use spread operator
const updatedUser = {
...user,
name: 'New Name'
}
const updatedArray = [...items, newItem]
// ❌ NEVER mutate directly
user.name = 'New Name' // BAD
items.push(newItem) // BAD
```
### Error Handling
```typescript
// ✅ GOOD: Comprehensive error handling
async function fetchData(url: string) {
try {
const response = await fetch(url)
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`)
}
return await response.json()
} catch (error) {
console.error('Fetch failed:', error)
throw new Error('Failed to fetch data')
}
}
// ❌ BAD: No error handling
async function fetchData(url) {
const response = await fetch(url)
return response.json()
}
```
### Async/Await Best Practices
```typescript
// ✅ GOOD: Parallel execution when possible
const [users, markets, stats] = await Promise.all([
fetchUsers(),
fetchMarkets(),
fetchStats()
])
// ❌ BAD: Sequential when unnecessary
const users = await fetchUsers()
const markets = await fetchMarkets()
const stats = await fetchStats()
```
### Type Safety
```typescript
// ✅ GOOD: Proper types
interface Market {
id: string
name: string
status: 'active' | 'resolved' | 'closed'
created_at: Date
}
function getMarket(id: string): Promise<Market> {
// Implementation
}
// ❌ BAD: Using 'any'
function getMarket(id: any): Promise<any> {
// Implementation
}
```
## React Best Practices
### Component Structure
```typescript
// ✅ GOOD: Functional component with types
interface ButtonProps {
children: React.ReactNode
onClick: () => void
disabled?: boolean
variant?: 'primary' | 'secondary'
}
export function Button({
children,
onClick,
disabled = false,
variant = 'primary'
}: ButtonProps) {
return (
<button
onClick={onClick}
disabled={disabled}
className={`btn btn-${variant}`}
>
{children}
</button>
)
}
// ❌ BAD: No types, unclear structure
export function Button(props) {
return <button onClick={props.onClick}>{props.children}</button>
}
```
### Custom Hooks
```typescript
// ✅ GOOD: Reusable custom hook
export function useDebounce<T>(value: T, delay: number): T {
const [debouncedValue, setDebouncedValue] = useState<T>(value)
useEffect(() => {
const handler = setTimeout(() => {
setDebouncedValue(value)
}, delay)
return () => clearTimeout(handler)
}, [value, delay])
return debouncedValue
}
// Usage
const debouncedQuery = useDebounce(searchQuery, 500)
```
### State Management
```typescript
// ✅ GOOD: Proper state updates
const [count, setCount] = useState(0)
// Functional update for state based on previous state
setCount(prev => prev + 1)
// ❌ BAD: Direct state reference
setCount(count + 1) // Can be stale in async scenarios
```
### Conditional Rendering
```typescript
// ✅ GOOD: Clear conditional rendering
{isLoading && <Spinner />}
{error && <ErrorMessage error={error} />}
{data && <DataDisplay data={data} />}
// ❌ BAD: Ternary hell
{isLoading ? <Spinner /> : error ? <ErrorMessage error={error} /> : data ? <DataDisplay data={data} /> : null}
```
## API Design Standards
### REST API Conventions
```
GET /api/markets # List all markets
GET /api/markets/:id # Get specific market
POST /api/markets # Create new market
PUT /api/markets/:id # Update market (full)
PATCH /api/markets/:id # Update market (partial)
DELETE /api/markets/:id # Delete market
# Query parameters for filtering
GET /api/markets?status=active&limit=10&offset=0
```
### Response Format
```typescript
// ✅ GOOD: Consistent response structure
interface ApiResponse<T> {
success: boolean
data?: T
error?: string
meta?: {
total: number
page: number
limit: number
}
}
// Success response
return NextResponse.json({
success: true,
data: markets,
meta: { total: 100, page: 1, limit: 10 }
})
// Error response
return NextResponse.json({
success: false,
error: 'Invalid request'
}, { status: 400 })
```
### Input Validation
```typescript
import { z } from 'zod'
// ✅ GOOD: Schema validation
const CreateMarketSchema = z.object({
name: z.string().min(1).max(200),
description: z.string().min(1).max(2000),
endDate: z.string().datetime(),
categories: z.array(z.string()).min(1)
})
export async function POST(request: Request) {
const body = await request.json()
try {
const validated = CreateMarketSchema.parse(body)
// Proceed with validated data
} catch (error) {
if (error instanceof z.ZodError) {
return NextResponse.json({
success: false,
error: 'Validation failed',
details: error.errors
}, { status: 400 })
}
}
}
```
## File Organization
### Project Structure
```
src/
├── app/ # Next.js App Router
│ ├── api/ # API routes
│ ├── markets/ # Market pages
│ └── (auth)/ # Auth pages (route groups)
├── components/ # React components
│ ├── ui/ # Generic UI components
│ ├── forms/ # Form components
│ └── layouts/ # Layout components
├── hooks/ # Custom React hooks
├── lib/ # Utilities and configs
│ ├── api/ # API clients
│ ├── utils/ # Helper functions
│ └── constants/ # Constants
├── types/ # TypeScript types
└── styles/ # Global styles
```
### File Naming
```
components/Button.tsx # PascalCase for components
hooks/useAuth.ts # camelCase with 'use' prefix
lib/formatDate.ts # camelCase for utilities
types/market.types.ts # camelCase with .types suffix
```
## Comments & Documentation
### When to Comment
```typescript
// ✅ GOOD: Explain WHY, not WHAT
// Use exponential backoff to avoid overwhelming the API during outages
const delay = Math.min(1000 * Math.pow(2, retryCount), 30000)
// Deliberately using mutation here for performance with large arrays
items.push(newItem)
// ❌ BAD: Stating the obvious
// Increment counter by 1
count++
// Set name to user's name
name = user.name
```
### JSDoc for Public APIs
```typescript
/**
* Searches markets using semantic similarity.
*
* @param query - Natural language search query
* @param limit - Maximum number of results (default: 10)
* @returns Array of markets sorted by similarity score
* @throws {Error} If OpenAI API fails or Redis unavailable
*
* @example
* ```typescript
* const results = await searchMarkets('election', 5)
* console.log(results[0].name) // "Trump vs Biden"
* ```
*/
export async function searchMarkets(
query: string,
limit: number = 10
): Promise<Market[]> {
// Implementation
}
```
## Performance Best Practices
### Memoization
```typescript
import { useMemo, useCallback } from 'react'
// ✅ GOOD: Memoize expensive computations
const sortedMarkets = useMemo(() => {
return markets.sort((a, b) => b.volume - a.volume)
}, [markets])
// ✅ GOOD: Memoize callbacks
const handleSearch = useCallback((query: string) => {
setSearchQuery(query)
}, [])
```
### Lazy Loading
```typescript
import { lazy, Suspense } from 'react'
// ✅ GOOD: Lazy load heavy components
const HeavyChart = lazy(() => import('./HeavyChart'))
export function Dashboard() {
return (
<Suspense fallback={<Spinner />}>
<HeavyChart />
</Suspense>
)
}
```
### Database Queries
```typescript
// ✅ GOOD: Select only needed columns
const { data } = await supabase
.from('markets')
.select('id, name, status')
.limit(10)
// ❌ BAD: Select everything
const { data } = await supabase
.from('markets')
.select('*')
```
## Testing Standards
### Test Structure (AAA Pattern)
```typescript
test('calculates similarity correctly', () => {
// Arrange
const vector1 = [1, 0, 0]
const vector2 = [0, 1, 0]
// Act
const similarity = calculateCosineSimilarity(vector1, vector2)
// Assert
expect(similarity).toBe(0)
})
```
### Test Naming
```typescript
// ✅ GOOD: Descriptive test names
test('returns empty array when no markets match query', () => { })
test('throws error when OpenAI API key is missing', () => { })
test('falls back to substring search when Redis unavailable', () => { })
// ❌ BAD: Vague test names
test('works', () => { })
test('test search', () => { })
```
## Code Smell Detection
Watch for these anti-patterns:
### 1. Long Functions
```typescript
// ❌ BAD: Function > 50 lines
function processMarketData() {
// 100 lines of code
}
// ✅ GOOD: Split into smaller functions
function processMarketData() {
const validated = validateData()
const transformed = transformData(validated)
return saveData(transformed)
}
```
### 2. Deep Nesting
```typescript
// ❌ BAD: 5+ levels of nesting
if (user) {
if (user.isAdmin) {
if (market) {
if (market.isActive) {
if (hasPermission) {
// Do something
}
}
}
}
}
// ✅ GOOD: Early returns
if (!user) return
if (!user.isAdmin) return
if (!market) return
if (!market.isActive) return
if (!hasPermission) return
// Do something
```
### 3. Magic Numbers
```typescript
// ❌ BAD: Unexplained numbers
if (retryCount > 3) { }
setTimeout(callback, 500)
// ✅ GOOD: Named constants
const MAX_RETRIES = 3
const DEBOUNCE_DELAY_MS = 500
if (retryCount > MAX_RETRIES) { }
setTimeout(callback, DEBOUNCE_DELAY_MS)
```
**Remember**: Code quality is not negotiable. Clear, maintainable code enables rapid development and confident refactoring.

298
skills/configure-ecc/SKILL.md Normal file
View File

@@ -0,0 +1,298 @@
---
name: configure-ecc
description: Interactive installer for Everything Claude Code — guides users through selecting and installing skills and rules to user-level or project-level directories, verifies paths, and optionally optimizes installed files.
---
# Configure Everything Claude Code (ECC)
An interactive, step-by-step installation wizard for the Everything Claude Code project. Uses `AskUserQuestion` to guide users through selective installation of skills and rules, then verifies correctness and offers optimization.
## When to Activate
- User says "configure ecc", "install ecc", "setup everything claude code", or similar
- User wants to selectively install skills or rules from this project
- User wants to verify or fix an existing ECC installation
- User wants to optimize installed skills or rules for their project
## Prerequisites
This skill must be accessible to Claude Code before activation. Two ways to bootstrap:
1. **Via Plugin**: `/plugin install everything-claude-code` — the plugin loads this skill automatically
2. **Manual**: Copy only this skill to `~/.claude/skills/configure-ecc/SKILL.md`, then activate by saying "configure ecc"
---
## Step 0: Clone ECC Repository
Before any installation, clone the latest ECC source to `/tmp`:
```bash
rm -rf /tmp/everything-claude-code
git clone https://github.com/affaan-m/everything-claude-code.git /tmp/everything-claude-code
```
Set `ECC_ROOT=/tmp/everything-claude-code` as the source for all subsequent copy operations.
If the clone fails (network issues, etc.), use `AskUserQuestion` to ask the user to provide a local path to an existing ECC clone.
---
## Step 1: Choose Installation Level
Use `AskUserQuestion` to ask the user where to install:
```
Question: "Where should ECC components be installed?"
Options:
- "User-level (~/.claude/)" — "Applies to all your Claude Code projects"
- "Project-level (.claude/)" — "Applies only to the current project"
- "Both" — "Common/shared items user-level, project-specific items project-level"
```
Store the choice as `INSTALL_LEVEL`. Set the target directory:
- User-level: `TARGET=~/.claude`
- Project-level: `TARGET=.claude` (relative to current project root)
- Both: `TARGET_USER=~/.claude`, `TARGET_PROJECT=.claude`
Create the target directories if they don't exist:
```bash
mkdir -p $TARGET/skills $TARGET/rules
```
---
## Step 2: Select & Install Skills
### 2a: Choose Skill Categories
There are 27 skills organized into 4 categories. Use `AskUserQuestion` with `multiSelect: true`:
```
Question: "Which skill categories do you want to install?"
Options:
- "Framework & Language" — "Django, Spring Boot, Go, Python, Java, Frontend, Backend patterns"
- "Database" — "PostgreSQL, ClickHouse, JPA/Hibernate patterns"
- "Workflow & Quality" — "TDD, verification, learning, security review, compaction"
- "All skills" — "Install every available skill"
```
### 2b: Confirm Individual Skills
For each selected category, print the full list of skills below and ask the user to confirm or deselect specific ones. If the list exceeds 4 items, print the list as text and use `AskUserQuestion` with an "Install all listed" option plus "Other" for the user to paste specific names.
**Category: Framework & Language (16 skills)**
| Skill | Description |
|-------|-------------|
| `backend-patterns` | Backend architecture, API design, server-side best practices for Node.js/Express/Next.js |
| `coding-standards` | Universal coding standards for TypeScript, JavaScript, React, Node.js |
| `django-patterns` | Django architecture, REST API with DRF, ORM, caching, signals, middleware |
| `django-security` | Django security: auth, CSRF, SQL injection, XSS prevention |
| `django-tdd` | Django testing with pytest-django, factory_boy, mocking, coverage |
| `django-verification` | Django verification loop: migrations, linting, tests, security scans |
| `frontend-patterns` | React, Next.js, state management, performance, UI patterns |
| `golang-patterns` | Idiomatic Go patterns, conventions for robust Go applications |
| `golang-testing` | Go testing: table-driven tests, subtests, benchmarks, fuzzing |
| `java-coding-standards` | Java coding standards for Spring Boot: naming, immutability, Optional, streams |
| `python-patterns` | Pythonic idioms, PEP 8, type hints, best practices |
| `python-testing` | Python testing with pytest, TDD, fixtures, mocking, parametrization |
| `springboot-patterns` | Spring Boot architecture, REST API, layered services, caching, async |
| `springboot-security` | Spring Security: authn/authz, validation, CSRF, secrets, rate limiting |
| `springboot-tdd` | Spring Boot TDD with JUnit 5, Mockito, MockMvc, Testcontainers |
| `springboot-verification` | Spring Boot verification: build, static analysis, tests, security scans |
**Category: Database (3 skills)**
| Skill | Description |
|-------|-------------|
| `clickhouse-io` | ClickHouse patterns, query optimization, analytics, data engineering |
| `jpa-patterns` | JPA/Hibernate entity design, relationships, query optimization, transactions |
| `postgres-patterns` | PostgreSQL query optimization, schema design, indexing, security |
**Category: Workflow & Quality (8 skills)**
| Skill | Description |
|-------|-------------|
| `continuous-learning` | Auto-extract reusable patterns from sessions as learned skills |
| `continuous-learning-v2` | Instinct-based learning with confidence scoring, evolves into skills/commands/agents |
| `eval-harness` | Formal evaluation framework for eval-driven development (EDD) |
| `iterative-retrieval` | Progressive context refinement for subagent context problem |
| `security-review` | Security checklist: auth, input, secrets, API, payment features |
| `strategic-compact` | Suggests manual context compaction at logical intervals |
| `tdd-workflow` | Enforces TDD with 80%+ coverage: unit, integration, E2E |
| `verification-loop` | Verification and quality loop patterns |
**Standalone**
| Skill | Description |
|-------|-------------|
| `project-guidelines-example` | Template for creating project-specific skills |
### 2c: Execute Installation
For each selected skill, copy the entire skill directory:
```bash
cp -r $ECC_ROOT/skills/<skill-name> $TARGET/skills/
```
Note: `continuous-learning` and `continuous-learning-v2` have extra files (config.json, hooks, scripts) — ensure the entire directory is copied, not just SKILL.md.
---
## Step 3: Select & Install Rules
Use `AskUserQuestion` with `multiSelect: true`:
```
Question: "Which rule sets do you want to install?"
Options:
- "Common rules (Recommended)" — "Language-agnostic principles: coding style, git workflow, testing, security, etc. (8 files)"
- "TypeScript/JavaScript" — "TS/JS patterns, hooks, testing with Playwright (5 files)"
- "Python" — "Python patterns, pytest, black/ruff formatting (5 files)"
- "Go" — "Go patterns, table-driven tests, gofmt/staticcheck (5 files)"
```
Execute installation:
```bash
# Common rules (flat copy into rules/)
cp -r $ECC_ROOT/rules/common/* $TARGET/rules/
# Language-specific rules (flat copy into rules/)
cp -r $ECC_ROOT/rules/typescript/* $TARGET/rules/ # if selected
cp -r $ECC_ROOT/rules/python/* $TARGET/rules/ # if selected
cp -r $ECC_ROOT/rules/golang/* $TARGET/rules/ # if selected
```
**Important**: If the user selects any language-specific rules but NOT common rules, warn them:
> "Language-specific rules extend the common rules. Installing without common rules may result in incomplete coverage. Install common rules too?"
---
## Step 4: Post-Installation Verification
After installation, perform these automated checks:
### 4a: Verify File Existence
List all installed files and confirm they exist at the target location:
```bash
ls -la $TARGET/skills/
ls -la $TARGET/rules/
```
### 4b: Check Path References
Scan all installed `.md` files for path references:
```bash
grep -rn "~/.claude/" $TARGET/skills/ $TARGET/rules/
grep -rn "../common/" $TARGET/rules/
grep -rn "skills/" $TARGET/skills/
```
**For project-level installs**, flag any references to `~/.claude/` paths:
- If a skill references `~/.claude/settings.json` — this is usually fine (settings are always user-level)
- If a skill references `~/.claude/skills/` or `~/.claude/rules/` — this may be broken if installed only at project level
- If a skill references another skill by name — check that the referenced skill was also installed
### 4c: Check Cross-References Between Skills
Some skills reference others. Verify these dependencies:
- `django-tdd` may reference `django-patterns`
- `springboot-tdd` may reference `springboot-patterns`
- `continuous-learning-v2` references `~/.claude/homunculus/` directory
- `python-testing` may reference `python-patterns`
- `golang-testing` may reference `golang-patterns`
- Language-specific rules reference `common/` counterparts
### 4d: Report Issues
For each issue found, report:
1. **File**: The file containing the problematic reference
2. **Line**: The line number
3. **Issue**: What's wrong (e.g., "references ~/.claude/skills/python-patterns but python-patterns was not installed")
4. **Suggested fix**: What to do (e.g., "install python-patterns skill" or "update path to .claude/skills/")
---
## Step 5: Optimize Installed Files (Optional)
Use `AskUserQuestion`:
```
Question: "Would you like to optimize the installed files for your project?"
Options:
- "Optimize skills" — "Remove irrelevant sections, adjust paths, tailor to your tech stack"
- "Optimize rules" — "Adjust coverage targets, add project-specific patterns, customize tool configs"
- "Optimize both" — "Full optimization of all installed files"
- "Skip" — "Keep everything as-is"
```
### If optimizing skills:
1. Read each installed SKILL.md
2. Ask the user what their project's tech stack is (if not already known)
3. For each skill, suggest removals of irrelevant sections
4. Edit the SKILL.md files in-place at the installation target (NOT the source repo)
5. Fix any path issues found in Step 4
### If optimizing rules:
1. Read each installed rule .md file
2. Ask the user about their preferences:
- Test coverage target (default 80%)
- Preferred formatting tools
- Git workflow conventions
- Security requirements
3. Edit the rule files in-place at the installation target
**Critical**: Only modify files in the installation target (`$TARGET/`), NEVER modify files in the source ECC repository (`$ECC_ROOT/`).
---
## Step 6: Installation Summary
Clean up the cloned repository from `/tmp`:
```bash
rm -rf /tmp/everything-claude-code
```
Then print a summary report:
```
## ECC Installation Complete
### Installation Target
- Level: [user-level / project-level / both]
- Path: [target path]
### Skills Installed ([count])
- skill-1, skill-2, skill-3, ...
### Rules Installed ([count])
- common (8 files)
- typescript (5 files)
- ...
### Verification Results
- [count] issues found, [count] fixed
- [list any remaining issues]
### Optimizations Applied
- [list changes made, or "None"]
```
---
## Troubleshooting
### "Skills not being picked up by Claude Code"
- Verify the skill directory contains a `SKILL.md` file (not just loose .md files)
- For user-level: check `~/.claude/skills/<skill-name>/SKILL.md` exists
- For project-level: check `.claude/skills/<skill-name>/SKILL.md` exists
### "Rules not working"
- Rules are flat files, not in subdirectories: `$TARGET/rules/coding-style.md` (correct) vs `$TARGET/rules/common/coding-style.md` (incorrect for flat install)
- Restart Claude Code after installing rules
### "Path reference errors after project-level install"
- Some skills assume `~/.claude/` paths. Run Step 4 verification to find and fix these.
- For `continuous-learning-v2`, the `~/.claude/homunculus/` directory is always user-level — this is expected and not an error.

292
skills/continuous-learning-v2/SKILL.md Normal file
View File

@@ -0,0 +1,292 @@
---
name: continuous-learning-v2
description: Instinct-based learning system that observes sessions via hooks, creates atomic instincts with confidence scoring, and evolves them into skills/commands/agents.
version: 2.0.0
---
# Continuous Learning v2 - Instinct-Based Architecture
An advanced learning system that turns your Claude Code sessions into reusable knowledge through atomic "instincts" - small learned behaviors with confidence scoring.
## When to Activate
- Setting up automatic learning from Claude Code sessions
- Configuring instinct-based behavior extraction via hooks
- Tuning confidence thresholds for learned behaviors
- Reviewing, exporting, or importing instinct libraries
- Evolving instincts into full skills, commands, or agents
## What's New in v2
| Feature | v1 | v2 |
|---------|----|----|
| Observation | Stop hook (session end) | PreToolUse/PostToolUse (100% reliable) |
| Analysis | Main context | Background agent (Haiku) |
| Granularity | Full skills | Atomic "instincts" |
| Confidence | None | 0.3-0.9 weighted |
| Evolution | Direct to skill | Instincts → cluster → skill/command/agent |
| Sharing | None | Export/import instincts |
## The Instinct Model
An instinct is a small learned behavior:
```yaml
---
id: prefer-functional-style
trigger: "when writing new functions"
confidence: 0.7
domain: "code-style"
source: "session-observation"
---
# Prefer Functional Style
## Action
Use functional patterns over classes when appropriate.
## Evidence
- Observed 5 instances of functional pattern preference
- User corrected class-based approach to functional on 2025-01-15
```
**Properties:**
- **Atomic** — one trigger, one action
- **Confidence-weighted** — 0.3 = tentative, 0.9 = near certain
- **Domain-tagged** — code-style, testing, git, debugging, workflow, etc.
- **Evidence-backed** — tracks what observations created it
## How It Works
```
Session Activity
│
│ Hooks capture prompts + tool use (100% reliable)
▼
┌─────────────────────────────────────────┐
│ observations.jsonl │
│ (prompts, tool calls, outcomes) │
└─────────────────────────────────────────┘
│
│ Observer agent reads (background, Haiku)
▼
┌─────────────────────────────────────────┐
│ PATTERN DETECTION │
│ • User corrections → instinct │
│ • Error resolutions → instinct │
│ • Repeated workflows → instinct │
└─────────────────────────────────────────┘
│
│ Creates/updates
▼
┌─────────────────────────────────────────┐
│ instincts/personal/ │
│ • prefer-functional.md (0.7) │
│ • always-test-first.md (0.9) │
│ • use-zod-validation.md (0.6) │
└─────────────────────────────────────────┘
│
│ /evolve clusters
▼
┌─────────────────────────────────────────┐
│ evolved/ │
│ • commands/new-feature.md │
│ • skills/testing-workflow.md │
│ • agents/refactor-specialist.md │
└─────────────────────────────────────────┘
```
## Quick Start
### 1. Enable Observation Hooks
Add to your `~/.claude/settings.json`.
**If installed as a plugin** (recommended):
```json
{
"hooks": {
"PreToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh pre"
}]
}],
"PostToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh post"
}]
}]
}
}
```
**If installed manually** to `~/.claude/skills`:
```json
{
"hooks": {
"PreToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh pre"
}]
}],
"PostToolUse": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh post"
}]
}]
}
}
```
### 2. Initialize Directory Structure
The Python CLI will create these automatically, but you can also create them manually:
```bash
mkdir -p ~/.claude/homunculus/{instincts/{personal,inherited},evolved/{agents,skills,commands}}
touch ~/.claude/homunculus/observations.jsonl
```
### 3. Use the Instinct Commands
```bash
/instinct-status # Show learned instincts with confidence scores
/evolve # Cluster related instincts into skills/commands
/instinct-export # Export instincts for sharing
/instinct-import # Import instincts from others
```
## Commands
| Command | Description |
|---------|-------------|
| `/instinct-status` | Show all learned instincts with confidence |
| `/evolve` | Cluster related instincts into skills/commands |
| `/instinct-export` | Export instincts for sharing |
| `/instinct-import <file>` | Import instincts from others |
## Configuration
Edit `config.json`:
```json
{
"version": "2.0",
"observation": {
"enabled": true,
"store_path": "~/.claude/homunculus/observations.jsonl",
"max_file_size_mb": 10,
"archive_after_days": 7
},
"instincts": {
"personal_path": "~/.claude/homunculus/instincts/personal/",
"inherited_path": "~/.claude/homunculus/instincts/inherited/",
"min_confidence": 0.3,
"auto_approve_threshold": 0.7,
"confidence_decay_rate": 0.05
},
"observer": {
"enabled": true,
"model": "haiku",
"run_interval_minutes": 5,
"patterns_to_detect": [
"user_corrections",
"error_resolutions",
"repeated_workflows",
"tool_preferences"
]
},
"evolution": {
"cluster_threshold": 3,
"evolved_path": "~/.claude/homunculus/evolved/"
}
}
```
## File Structure
```
~/.claude/homunculus/
├── identity.json # Your profile, technical level
├── observations.jsonl # Current session observations
├── observations.archive/ # Processed observations
├── instincts/
│ ├── personal/ # Auto-learned instincts
│ └── inherited/ # Imported from others
└── evolved/
├── agents/ # Generated specialist agents
├── skills/ # Generated skills
└── commands/ # Generated commands
```
## Integration with Skill Creator
When you use the [Skill Creator GitHub App](https://skill-creator.app), it now generates **both**:
- Traditional SKILL.md files (for backward compatibility)
- Instinct collections (for v2 learning system)
Instincts from repo analysis have `source: "repo-analysis"` and include the source repository URL.
## Confidence Scoring
Confidence evolves over time:
| Score | Meaning | Behavior |
|-------|---------|----------|
| 0.3 | Tentative | Suggested but not enforced |
| 0.5 | Moderate | Applied when relevant |
| 0.7 | Strong | Auto-approved for application |
| 0.9 | Near-certain | Core behavior |
**Confidence increases** when:
- Pattern is repeatedly observed
- User doesn't correct the suggested behavior
- Similar instincts from other sources agree
**Confidence decreases** when:
- User explicitly corrects the behavior
- Pattern isn't observed for extended periods
- Contradicting evidence appears
## Why Hooks vs Skills for Observation?
> "v1 relied on skills to observe. Skills are probabilistic—they fire ~50-80% of the time based on Claude's judgment."
Hooks fire **100% of the time**, deterministically. This means:
- Every tool call is observed
- No patterns are missed
- Learning is comprehensive
## Backward Compatibility
v2 is fully compatible with v1:
- Existing `~/.claude/skills/learned/` skills still work
- Stop hook still runs (but now also feeds into v2)
- Gradual migration path: run both in parallel
## Privacy
- Observations stay **local** on your machine
- Only **instincts** (patterns) can be exported
- No actual code or conversation content is shared
- You control what gets exported
## Related
- [Skill Creator](https://skill-creator.app) - Generate instincts from repo history
- [Homunculus](https://github.com/humanplane/homunculus) - Inspiration for v2 architecture
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Continuous learning section
---
*Instinct-based learning: teaching Claude your patterns, one observation at a time.*

137
skills/continuous-learning-v2/agents/observer.md Normal file
View File

@@ -0,0 +1,137 @@
---
name: observer
description: Background agent that analyzes session observations to detect patterns and create instincts. Uses Haiku for cost-efficiency.
model: haiku
run_mode: background
---
# Observer Agent
A background agent that analyzes observations from Claude Code sessions to detect patterns and create instincts.
## When to Run
- After significant session activity (20+ tool calls)
- When user runs `/analyze-patterns`
- On a scheduled interval (configurable, default 5 minutes)
- When triggered by observation hook (SIGUSR1)
## Input
Reads observations from `~/.claude/homunculus/observations.jsonl`:
```jsonl
{"timestamp":"2025-01-22T10:30:00Z","event":"tool_start","session":"abc123","tool":"Edit","input":"..."}
{"timestamp":"2025-01-22T10:30:01Z","event":"tool_complete","session":"abc123","tool":"Edit","output":"..."}
{"timestamp":"2025-01-22T10:30:05Z","event":"tool_start","session":"abc123","tool":"Bash","input":"npm test"}
{"timestamp":"2025-01-22T10:30:10Z","event":"tool_complete","session":"abc123","tool":"Bash","output":"All tests pass"}
```
## Pattern Detection
Look for these patterns in observations:
### 1. User Corrections
When a user's follow-up message corrects Claude's previous action:
- "No, use X instead of Y"
- "Actually, I meant..."
- Immediate undo/redo patterns
→ Create instinct: "When doing X, prefer Y"
### 2. Error Resolutions
When an error is followed by a fix:
- Tool output contains error
- Next few tool calls fix it
- Same error type resolved similarly multiple times
→ Create instinct: "When encountering error X, try Y"
### 3. Repeated Workflows
When the same sequence of tools is used multiple times:
- Same tool sequence with similar inputs
- File patterns that change together
- Time-clustered operations
→ Create workflow instinct: "When doing X, follow steps Y, Z, W"
### 4. Tool Preferences
When certain tools are consistently preferred:
- Always uses Grep before Edit
- Prefers Read over Bash cat
- Uses specific Bash commands for certain tasks
→ Create instinct: "When needing X, use tool Y"
## Output
Creates/updates instincts in `~/.claude/homunculus/instincts/personal/`:
```yaml
---
id: prefer-grep-before-edit
trigger: "when searching for code to modify"
confidence: 0.65
domain: "workflow"
source: "session-observation"
---
# Prefer Grep Before Edit
## Action
Always use Grep to find the exact location before using Edit.
## Evidence
- Observed 8 times in session abc123
- Pattern: Grep → Read → Edit sequence
- Last observed: 2025-01-22
```
## Confidence Calculation
Initial confidence based on observation frequency:
- 1-2 observations: 0.3 (tentative)
- 3-5 observations: 0.5 (moderate)
- 6-10 observations: 0.7 (strong)
- 11+ observations: 0.85 (very strong)
Confidence adjusts over time:
- +0.05 for each confirming observation
- -0.1 for each contradicting observation
- -0.02 per week without observation (decay)
## Important Guidelines
1. **Be Conservative**: Only create instincts for clear patterns (3+ observations)
2. **Be Specific**: Narrow triggers are better than broad ones
3. **Track Evidence**: Always include what observations led to the instinct
4. **Respect Privacy**: Never include actual code snippets, only patterns
5. **Merge Similar**: If a new instinct is similar to existing, update rather than duplicate
## Example Analysis Session
Given observations:
```jsonl
{"event":"tool_start","tool":"Grep","input":"pattern: useState"}
{"event":"tool_complete","tool":"Grep","output":"Found in 3 files"}
{"event":"tool_start","tool":"Read","input":"src/hooks/useAuth.ts"}
{"event":"tool_complete","tool":"Read","output":"[file content]"}
{"event":"tool_start","tool":"Edit","input":"src/hooks/useAuth.ts..."}
```
Analysis:
- Detected workflow: Grep → Read → Edit
- Frequency: Seen 5 times this session
- Create instinct:
- trigger: "when modifying code"
- action: "Search with Grep, confirm with Read, then Edit"
- confidence: 0.6
- domain: "workflow"
## Integration with Skill Creator
When instincts are imported from Skill Creator (repo analysis), they have:
- `source: "repo-analysis"`
- `source_repo: "https://github.com/..."`
These should be treated as team/project conventions with higher initial confidence (0.7+).

143
skills/continuous-learning-v2/agents/start-observer.sh Executable file
View File

@@ -0,0 +1,143 @@
#!/bin/bash
# Continuous Learning v2 - Observer Agent Launcher
#
# Starts the background observer agent that analyzes observations
# and creates instincts. Uses Haiku model for cost efficiency.
#
# Usage:
# start-observer.sh # Start observer in background
# start-observer.sh stop # Stop running observer
# start-observer.sh status # Check if observer is running
set -e
CONFIG_DIR="${HOME}/.claude/homunculus"
PID_FILE="${CONFIG_DIR}/.observer.pid"
LOG_FILE="${CONFIG_DIR}/observer.log"
OBSERVATIONS_FILE="${CONFIG_DIR}/observations.jsonl"
mkdir -p "$CONFIG_DIR"
case "${1:-start}" in
stop)
if [ -f "$PID_FILE" ]; then
pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
echo "Stopping observer (PID: $pid)..."
kill "$pid"
rm -f "$PID_FILE"
echo "Observer stopped."
else
echo "Observer not running (stale PID file)."
rm -f "$PID_FILE"
fi
else
echo "Observer not running."
fi
exit 0
;;
status)
if [ -f "$PID_FILE" ]; then
pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
echo "Observer is running (PID: $pid)"
echo "Log: $LOG_FILE"
echo "Observations: $(wc -l < "$OBSERVATIONS_FILE" 2>/dev/null || echo 0) lines"
exit 0
else
echo "Observer not running (stale PID file)"
rm -f "$PID_FILE"
exit 1
fi
else
echo "Observer not running"
exit 1
fi
;;
start)
# Check if already running
if [ -f "$PID_FILE" ]; then
pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
echo "Observer already running (PID: $pid)"
exit 0
fi
rm -f "$PID_FILE"
fi
echo "Starting observer agent..."
# The observer loop
(
trap 'rm -f "$PID_FILE"; exit 0' TERM INT
analyze_observations() {
# Only analyze if observations file exists and has enough entries
if [ ! -f "$OBSERVATIONS_FILE" ]; then
return
fi
obs_count=$(wc -l < "$OBSERVATIONS_FILE" 2>/dev/null || echo 0)
if [ "$obs_count" -lt 10 ]; then
return
fi
echo "[$(date)] Analyzing $obs_count observations..." >> "$LOG_FILE"
# Use Claude Code with Haiku to analyze observations
# This spawns a quick analysis session
if command -v claude &> /dev/null; then
exit_code=0
claude --model haiku --max-turns 3 --print \
"Read $OBSERVATIONS_FILE and identify patterns. If you find 3+ occurrences of the same pattern, create an instinct file in $CONFIG_DIR/instincts/personal/ following the format in the observer agent spec. Be conservative - only create instincts for clear patterns." \
>> "$LOG_FILE" 2>&1 || exit_code=$?
if [ "$exit_code" -ne 0 ]; then
echo "[$(date)] Claude analysis failed (exit $exit_code)" >> "$LOG_FILE"
fi
else
echo "[$(date)] claude CLI not found, skipping analysis" >> "$LOG_FILE"
fi
# Archive processed observations
if [ -f "$OBSERVATIONS_FILE" ]; then
archive_dir="${CONFIG_DIR}/observations.archive"
mkdir -p "$archive_dir"
mv "$OBSERVATIONS_FILE" "$archive_dir/processed-$(date +%Y%m%d-%H%M%S).jsonl" 2>/dev/null || true
touch "$OBSERVATIONS_FILE"
fi
}
# Handle SIGUSR1 for on-demand analysis
trap 'analyze_observations' USR1
echo "$$" > "$PID_FILE"
echo "[$(date)] Observer started (PID: $$)" >> "$LOG_FILE"
while true; do
# Check every 5 minutes
sleep 300
analyze_observations
done
) &
disown
# Wait a moment for PID file
sleep 1
if [ -f "$PID_FILE" ]; then
echo "Observer started (PID: $(cat "$PID_FILE"))"
echo "Log: $LOG_FILE"
else
echo "Failed to start observer"
exit 1
fi
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
;;
esac

41
skills/continuous-learning-v2/config.json Normal file
View File

@@ -0,0 +1,41 @@
{
"version": "2.0",
"observation": {
"enabled": true,
"store_path": "~/.claude/homunculus/observations.jsonl",
"max_file_size_mb": 10,
"archive_after_days": 7,
"capture_tools": ["Edit", "Write", "Bash", "Read", "Grep", "Glob"],
"ignore_tools": ["TodoWrite"]
},
"instincts": {
"personal_path": "~/.claude/homunculus/instincts/personal/",
"inherited_path": "~/.claude/homunculus/instincts/inherited/",
"min_confidence": 0.3,
"auto_approve_threshold": 0.7,
"confidence_decay_rate": 0.02,
"max_instincts": 100
},
"observer": {
"enabled": false,
"model": "haiku",
"run_interval_minutes": 5,
"min_observations_to_analyze": 20,
"patterns_to_detect": [
"user_corrections",
"error_resolutions",
"repeated_workflows",
"tool_preferences",
"file_patterns"
]
},
"evolution": {
"cluster_threshold": 3,
"evolved_path": "~/.claude/homunculus/evolved/",
"auto_evolve": false
},
"integration": {
"skill_creator_api": "https://skill-creator.app/api",
"backward_compatible_v1": true
}
}

155
skills/continuous-learning-v2/hooks/observe.sh Executable file
View File

@@ -0,0 +1,155 @@
#!/bin/bash
# Continuous Learning v2 - Observation Hook
#
# Captures tool use events for pattern analysis.
# Claude Code passes hook data via stdin as JSON.
#
# Hook config (in ~/.claude/settings.json):
#
# If installed as a plugin, use ${CLAUDE_PLUGIN_ROOT}:
# {
# "hooks": {
# "PreToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh pre" }]
# }],
# "PostToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh post" }]
# }]
# }
# }
#
# If installed manually to ~/.claude/skills:
# {
# "hooks": {
# "PreToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh pre" }]
# }],
# "PostToolUse": [{
# "matcher": "*",
# "hooks": [{ "type": "command", "command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh post" }]
# }]
# }
# }
set -e
CONFIG_DIR="${HOME}/.claude/homunculus"
OBSERVATIONS_FILE="${CONFIG_DIR}/observations.jsonl"
MAX_FILE_SIZE_MB=10
# Ensure directory exists
mkdir -p "$CONFIG_DIR"
# Skip if disabled
if [ -f "$CONFIG_DIR/disabled" ]; then
exit 0
fi
# Read JSON from stdin (Claude Code hook format)
INPUT_JSON=$(cat)
# Exit if no input
if [ -z "$INPUT_JSON" ]; then
exit 0
fi
# Parse using python via stdin pipe (safe for all JSON payloads)
PARSED=$(echo "$INPUT_JSON" | python3 -c '
import json
import sys
try:
data = json.load(sys.stdin)
# Extract fields - Claude Code hook format
hook_type = data.get("hook_type", "unknown") # PreToolUse or PostToolUse
tool_name = data.get("tool_name", data.get("tool", "unknown"))
tool_input = data.get("tool_input", data.get("input", {}))
tool_output = data.get("tool_output", data.get("output", ""))
session_id = data.get("session_id", "unknown")
# Truncate large inputs/outputs
if isinstance(tool_input, dict):
tool_input_str = json.dumps(tool_input)[:5000]
else:
tool_input_str = str(tool_input)[:5000]
if isinstance(tool_output, dict):
tool_output_str = json.dumps(tool_output)[:5000]
else:
tool_output_str = str(tool_output)[:5000]
# Determine event type
event = "tool_start" if "Pre" in hook_type else "tool_complete"
print(json.dumps({
"parsed": True,
"event": event,
"tool": tool_name,
"input": tool_input_str if event == "tool_start" else None,
"output": tool_output_str if event == "tool_complete" else None,
"session": session_id
}))
except Exception as e:
print(json.dumps({"parsed": False, "error": str(e)}))
')
# Check if parsing succeeded
PARSED_OK=$(echo "$PARSED" | python3 -c "import json,sys; print(json.load(sys.stdin).get('parsed', False))")
if [ "$PARSED_OK" != "True" ]; then
# Fallback: log raw input for debugging
timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
TIMESTAMP="$timestamp" echo "$INPUT_JSON" | python3 -c "
import json, sys, os
raw = sys.stdin.read()[:2000]
print(json.dumps({'timestamp': os.environ['TIMESTAMP'], 'event': 'parse_error', 'raw': raw}))
" >> "$OBSERVATIONS_FILE"
exit 0
fi
# Archive if file too large
if [ -f "$OBSERVATIONS_FILE" ]; then
file_size_mb=$(du -m "$OBSERVATIONS_FILE" 2>/dev/null | cut -f1)
if [ "${file_size_mb:-0}" -ge "$MAX_FILE_SIZE_MB" ]; then
archive_dir="${CONFIG_DIR}/observations.archive"
mkdir -p "$archive_dir"
mv "$OBSERVATIONS_FILE" "$archive_dir/observations-$(date +%Y%m%d-%H%M%S).jsonl"
fi
fi
# Build and write observation
timestamp=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
TIMESTAMP="$timestamp" echo "$PARSED" | python3 -c "
import json, sys, os
parsed = json.load(sys.stdin)
observation = {
'timestamp': os.environ['TIMESTAMP'],
'event': parsed['event'],
'tool': parsed['tool'],
'session': parsed['session']
}
if parsed['input']:
observation['input'] = parsed['input']
if parsed['output']:
observation['output'] = parsed['output']
print(json.dumps(observation))
" >> "$OBSERVATIONS_FILE"
# Signal observer if running
OBSERVER_PID_FILE="${CONFIG_DIR}/.observer.pid"
if [ -f "$OBSERVER_PID_FILE" ]; then
observer_pid=$(cat "$OBSERVER_PID_FILE")
if kill -0 "$observer_pid" 2>/dev/null; then
kill -USR1 "$observer_pid" 2>/dev/null || true
fi
fi
exit 0

575
skills/continuous-learning-v2/scripts/instinct-cli.py Executable file
View File

@@ -0,0 +1,575 @@
#!/usr/bin/env python3
"""
Instinct CLI - Manage instincts for Continuous Learning v2
Commands:
status - Show all instincts and their status
import - Import instincts from file or URL
export - Export instincts to file
evolve - Cluster instincts into skills/commands/agents
"""
import argparse
import json
import os
import sys
import re
import urllib.request
from pathlib import Path
from datetime import datetime
from collections import defaultdict
from typing import Optional
# ─────────────────────────────────────────────
# Configuration
# ─────────────────────────────────────────────
HOMUNCULUS_DIR = Path.home() / ".claude" / "homunculus"
INSTINCTS_DIR = HOMUNCULUS_DIR / "instincts"
PERSONAL_DIR = INSTINCTS_DIR / "personal"
INHERITED_DIR = INSTINCTS_DIR / "inherited"
EVOLVED_DIR = HOMUNCULUS_DIR / "evolved"
OBSERVATIONS_FILE = HOMUNCULUS_DIR / "observations.jsonl"
# Ensure directories exist
for d in [PERSONAL_DIR, INHERITED_DIR, EVOLVED_DIR / "skills", EVOLVED_DIR / "commands", EVOLVED_DIR / "agents"]:
d.mkdir(parents=True, exist_ok=True)
# ─────────────────────────────────────────────
# Instinct Parser
# ─────────────────────────────────────────────
def parse_instinct_file(content: str) -> list[dict]:
"""Parse YAML-like instinct file format."""
instincts = []
current = {}
in_frontmatter = False
content_lines = []
for line in content.split('\n'):
if line.strip() == '---':
if in_frontmatter:
# End of frontmatter - content comes next, don't append yet
in_frontmatter = False
else:
# Start of frontmatter
in_frontmatter = True
if current:
current['content'] = '\n'.join(content_lines).strip()
instincts.append(current)
current = {}
content_lines = []
elif in_frontmatter:
# Parse YAML-like frontmatter
if ':' in line:
key, value = line.split(':', 1)
key = key.strip()
value = value.strip().strip('"').strip("'")
if key == 'confidence':
current[key] = float(value)
else:
current[key] = value
else:
content_lines.append(line)
# Don't forget the last instinct
if current:
current['content'] = '\n'.join(content_lines).strip()
instincts.append(current)
return [i for i in instincts if i.get('id')]
def load_all_instincts() -> list[dict]:
"""Load all instincts from personal and inherited directories."""
instincts = []
for directory in [PERSONAL_DIR, INHERITED_DIR]:
if not directory.exists():
continue
yaml_files = sorted(
set(directory.glob("*.yaml"))
| set(directory.glob("*.yml"))
| set(directory.glob("*.md"))
)
for file in yaml_files:
try:
content = file.read_text()
parsed = parse_instinct_file(content)
for inst in parsed:
inst['_source_file'] = str(file)
inst['_source_type'] = directory.name
instincts.extend(parsed)
except Exception as e:
print(f"Warning: Failed to parse {file}: {e}", file=sys.stderr)
return instincts
# ─────────────────────────────────────────────
# Status Command
# ─────────────────────────────────────────────
def cmd_status(args):
"""Show status of all instincts."""
instincts = load_all_instincts()
if not instincts:
print("No instincts found.")
print(f"\nInstinct directories:")
print(f" Personal: {PERSONAL_DIR}")
print(f" Inherited: {INHERITED_DIR}")
return
# Group by domain
by_domain = defaultdict(list)
for inst in instincts:
domain = inst.get('domain', 'general')
by_domain[domain].append(inst)
# Print header
print(f"\n{'='*60}")
print(f" INSTINCT STATUS - {len(instincts)} total")
print(f"{'='*60}\n")
# Summary by source
personal = [i for i in instincts if i.get('_source_type') == 'personal']
inherited = [i for i in instincts if i.get('_source_type') == 'inherited']
print(f" Personal: {len(personal)}")
print(f" Inherited: {len(inherited)}")
print()
# Print by domain
for domain in sorted(by_domain.keys()):
domain_instincts = by_domain[domain]
print(f"## {domain.upper()} ({len(domain_instincts)})")
print()
for inst in sorted(domain_instincts, key=lambda x: -x.get('confidence', 0.5)):
conf = inst.get('confidence', 0.5)
conf_bar = '█' * int(conf * 10) + '░' * (10 - int(conf * 10))
trigger = inst.get('trigger', 'unknown trigger')
source = inst.get('source', 'unknown')
print(f" {conf_bar} {int(conf*100):3d}% {inst.get('id', 'unnamed')}")
print(f" trigger: {trigger}")
# Extract action from content
content = inst.get('content', '')
action_match = re.search(r'## Action\s*\n\s*(.+?)(?:\n\n|\n##|$)', content, re.DOTALL)
if action_match:
action = action_match.group(1).strip().split('\n')[0]
print(f" action: {action[:60]}{'...' if len(action) > 60 else ''}")
print()
# Observations stats
if OBSERVATIONS_FILE.exists():
obs_count = sum(1 for _ in open(OBSERVATIONS_FILE))
print(f"─────────────────────────────────────────────────────────")
print(f" Observations: {obs_count} events logged")
print(f" File: {OBSERVATIONS_FILE}")
print(f"\n{'='*60}\n")
# ─────────────────────────────────────────────
# Import Command
# ─────────────────────────────────────────────
def cmd_import(args):
"""Import instincts from file or URL."""
source = args.source
# Fetch content
if source.startswith('http://') or source.startswith('https://'):
print(f"Fetching from URL: {source}")
try:
with urllib.request.urlopen(source) as response:
content = response.read().decode('utf-8')
except Exception as e:
print(f"Error fetching URL: {e}", file=sys.stderr)
return 1
else:
path = Path(source).expanduser()
if not path.exists():
print(f"File not found: {path}", file=sys.stderr)
return 1
content = path.read_text()
# Parse instincts
new_instincts = parse_instinct_file(content)
if not new_instincts:
print("No valid instincts found in source.")
return 1
print(f"\nFound {len(new_instincts)} instincts to import.\n")
# Load existing
existing = load_all_instincts()
existing_ids = {i.get('id') for i in existing}
# Categorize
to_add = []
duplicates = []
to_update = []
for inst in new_instincts:
inst_id = inst.get('id')
if inst_id in existing_ids:
# Check if we should update
existing_inst = next((e for e in existing if e.get('id') == inst_id), None)
if existing_inst:
if inst.get('confidence', 0) > existing_inst.get('confidence', 0):
to_update.append(inst)
else:
duplicates.append(inst)
else:
to_add.append(inst)
# Filter by minimum confidence
min_conf = args.min_confidence or 0.0
to_add = [i for i in to_add if i.get('confidence', 0.5) >= min_conf]
to_update = [i for i in to_update if i.get('confidence', 0.5) >= min_conf]
# Display summary
if to_add:
print(f"NEW ({len(to_add)}):")
for inst in to_add:
print(f" + {inst.get('id')} (confidence: {inst.get('confidence', 0.5):.2f})")
if to_update:
print(f"\nUPDATE ({len(to_update)}):")
for inst in to_update:
print(f" ~ {inst.get('id')} (confidence: {inst.get('confidence', 0.5):.2f})")
if duplicates:
print(f"\nSKIP ({len(duplicates)} - already exists with equal/higher confidence):")
for inst in duplicates[:5]:
print(f" - {inst.get('id')}")
if len(duplicates) > 5:
print(f" ... and {len(duplicates) - 5} more")
if args.dry_run:
print("\n[DRY RUN] No changes made.")
return 0
if not to_add and not to_update:
print("\nNothing to import.")
return 0
# Confirm
if not args.force:
response = input(f"\nImport {len(to_add)} new, update {len(to_update)}? [y/N] ")
if response.lower() != 'y':
print("Cancelled.")
return 0
# Write to inherited directory
timestamp = datetime.now().strftime('%Y%m%d-%H%M%S')
source_name = Path(source).stem if not source.startswith('http') else 'web-import'
output_file = INHERITED_DIR / f"{source_name}-{timestamp}.yaml"
all_to_write = to_add + to_update
output_content = f"# Imported from {source}\n# Date: {datetime.now().isoformat()}\n\n"
for inst in all_to_write:
output_content += "---\n"
output_content += f"id: {inst.get('id')}\n"
output_content += f"trigger: \"{inst.get('trigger', 'unknown')}\"\n"
output_content += f"confidence: {inst.get('confidence', 0.5)}\n"
output_content += f"domain: {inst.get('domain', 'general')}\n"
output_content += f"source: inherited\n"
output_content += f"imported_from: \"{source}\"\n"
if inst.get('source_repo'):
output_content += f"source_repo: {inst.get('source_repo')}\n"
output_content += "---\n\n"
output_content += inst.get('content', '') + "\n\n"
output_file.write_text(output_content)
print(f"\n✅ Import complete!")
print(f" Added: {len(to_add)}")
print(f" Updated: {len(to_update)}")
print(f" Saved to: {output_file}")
return 0
# ─────────────────────────────────────────────
# Export Command
# ─────────────────────────────────────────────
def cmd_export(args):
"""Export instincts to file."""
instincts = load_all_instincts()
if not instincts:
print("No instincts to export.")
return 1
# Filter by domain if specified
if args.domain:
instincts = [i for i in instincts if i.get('domain') == args.domain]
# Filter by minimum confidence
if args.min_confidence:
instincts = [i for i in instincts if i.get('confidence', 0.5) >= args.min_confidence]
if not instincts:
print("No instincts match the criteria.")
return 1
# Generate output
output = f"# Instincts export\n# Date: {datetime.now().isoformat()}\n# Total: {len(instincts)}\n\n"
for inst in instincts:
output += "---\n"
for key in ['id', 'trigger', 'confidence', 'domain', 'source', 'source_repo']:
if inst.get(key):
value = inst[key]
if key == 'trigger':
output += f'{key}: "{value}"\n'
else:
output += f"{key}: {value}\n"
output += "---\n\n"
output += inst.get('content', '') + "\n\n"
# Write to file or stdout
if args.output:
Path(args.output).write_text(output)
print(f"Exported {len(instincts)} instincts to {args.output}")
else:
print(output)
return 0
# ─────────────────────────────────────────────
# Evolve Command
# ─────────────────────────────────────────────
def cmd_evolve(args):
"""Analyze instincts and suggest evolutions to skills/commands/agents."""
instincts = load_all_instincts()
if len(instincts) < 3:
print("Need at least 3 instincts to analyze patterns.")
print(f"Currently have: {len(instincts)}")
return 1
print(f"\n{'='*60}")
print(f" EVOLVE ANALYSIS - {len(instincts)} instincts")
print(f"{'='*60}\n")
# Group by domain
by_domain = defaultdict(list)
for inst in instincts:
domain = inst.get('domain', 'general')
by_domain[domain].append(inst)
# High-confidence instincts by domain (candidates for skills)
high_conf = [i for i in instincts if i.get('confidence', 0) >= 0.8]
print(f"High confidence instincts (>=80%): {len(high_conf)}")
# Find clusters (instincts with similar triggers)
trigger_clusters = defaultdict(list)
for inst in instincts:
trigger = inst.get('trigger', '')
# Normalize trigger
trigger_key = trigger.lower()
for keyword in ['when', 'creating', 'writing', 'adding', 'implementing', 'testing']:
trigger_key = trigger_key.replace(keyword, '').strip()
trigger_clusters[trigger_key].append(inst)
# Find clusters with 3+ instincts (good skill candidates)
skill_candidates = []
for trigger, cluster in trigger_clusters.items():
if len(cluster) >= 2:
avg_conf = sum(i.get('confidence', 0.5) for i in cluster) / len(cluster)
skill_candidates.append({
'trigger': trigger,
'instincts': cluster,
'avg_confidence': avg_conf,
'domains': list(set(i.get('domain', 'general') for i in cluster))
})
# Sort by cluster size and confidence
skill_candidates.sort(key=lambda x: (-len(x['instincts']), -x['avg_confidence']))
print(f"\nPotential skill clusters found: {len(skill_candidates)}")
if skill_candidates:
print(f"\n## SKILL CANDIDATES\n")
for i, cand in enumerate(skill_candidates[:5], 1):
print(f"{i}. Cluster: \"{cand['trigger']}\"")
print(f" Instincts: {len(cand['instincts'])}")
print(f" Avg confidence: {cand['avg_confidence']:.0%}")
print(f" Domains: {', '.join(cand['domains'])}")
print(f" Instincts:")
for inst in cand['instincts'][:3]:
print(f" - {inst.get('id')}")
print()
# Command candidates (workflow instincts with high confidence)
workflow_instincts = [i for i in instincts if i.get('domain') == 'workflow' and i.get('confidence', 0) >= 0.7]
if workflow_instincts:
print(f"\n## COMMAND CANDIDATES ({len(workflow_instincts)})\n")
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
# Suggest command name
cmd_name = trigger.replace('when ', '').replace('implementing ', '').replace('a ', '')
cmd_name = cmd_name.replace(' ', '-')[:20]
print(f" /{cmd_name}")
print(f" From: {inst.get('id')}")
print(f" Confidence: {inst.get('confidence', 0.5):.0%}")
print()
# Agent candidates (complex multi-step patterns)
agent_candidates = [c for c in skill_candidates if len(c['instincts']) >= 3 and c['avg_confidence'] >= 0.75]
if agent_candidates:
print(f"\n## AGENT CANDIDATES ({len(agent_candidates)})\n")
for cand in agent_candidates[:3]:
agent_name = cand['trigger'].replace(' ', '-')[:20] + '-agent'
print(f" {agent_name}")
print(f" Covers {len(cand['instincts'])} instincts")
print(f" Avg confidence: {cand['avg_confidence']:.0%}")
print()
if args.generate:
generated = _generate_evolved(skill_candidates, workflow_instincts, agent_candidates)
if generated:
print(f"\n✅ Generated {len(generated)} evolved structures:")
for path in generated:
print(f" {path}")
else:
print("\nNo structures generated (need higher-confidence clusters).")
print(f"\n{'='*60}\n")
return 0
# ─────────────────────────────────────────────
# Generate Evolved Structures
# ─────────────────────────────────────────────
def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_candidates: list) -> list[str]:
"""Generate skill/command/agent files from analyzed instinct clusters."""
generated = []
# Generate skills from top candidates
for cand in skill_candidates[:5]:
trigger = cand['trigger'].strip()
if not trigger:
continue
name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:30]
if not name:
continue
skill_dir = EVOLVED_DIR / "skills" / name
skill_dir.mkdir(parents=True, exist_ok=True)
content = f"# {name}\n\n"
content += f"Evolved from {len(cand['instincts'])} instincts "
content += f"(avg confidence: {cand['avg_confidence']:.0%})\n\n"
content += f"## When to Apply\n\n"
content += f"Trigger: {trigger}\n\n"
content += f"## Actions\n\n"
for inst in cand['instincts']:
inst_content = inst.get('content', '')
action_match = re.search(r'## Action\s*\n\s*(.+?)(?:\n\n|\n##|$)', inst_content, re.DOTALL)
action = action_match.group(1).strip() if action_match else inst.get('id', 'unnamed')
content += f"- {action}\n"
(skill_dir / "SKILL.md").write_text(content)
generated.append(str(skill_dir / "SKILL.md"))
# Generate commands from workflow instincts
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
cmd_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower().replace('when ', '').replace('implementing ', ''))
cmd_name = cmd_name.strip('-')[:20]
if not cmd_name:
continue
cmd_file = EVOLVED_DIR / "commands" / f"{cmd_name}.md"
content = f"# {cmd_name}\n\n"
content += f"Evolved from instinct: {inst.get('id', 'unnamed')}\n"
content += f"Confidence: {inst.get('confidence', 0.5):.0%}\n\n"
content += inst.get('content', '')
cmd_file.write_text(content)
generated.append(str(cmd_file))
# Generate agents from complex clusters
for cand in agent_candidates[:3]:
trigger = cand['trigger'].strip()
agent_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:20]
if not agent_name:
continue
agent_file = EVOLVED_DIR / "agents" / f"{agent_name}.md"
domains = ', '.join(cand['domains'])
instinct_ids = [i.get('id', 'unnamed') for i in cand['instincts']]
content = f"---\nmodel: sonnet\ntools: Read, Grep, Glob\n---\n"
content += f"# {agent_name}\n\n"
content += f"Evolved from {len(cand['instincts'])} instincts "
content += f"(avg confidence: {cand['avg_confidence']:.0%})\n"
content += f"Domains: {domains}\n\n"
content += f"## Source Instincts\n\n"
for iid in instinct_ids:
content += f"- {iid}\n"
agent_file.write_text(content)
generated.append(str(agent_file))
return generated
# ─────────────────────────────────────────────
# Main
# ─────────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(description='Instinct CLI for Continuous Learning v2')
subparsers = parser.add_subparsers(dest='command', help='Available commands')
# Status
status_parser = subparsers.add_parser('status', help='Show instinct status')
# Import
import_parser = subparsers.add_parser('import', help='Import instincts')
import_parser.add_argument('source', help='File path or URL')
import_parser.add_argument('--dry-run', action='store_true', help='Preview without importing')
import_parser.add_argument('--force', action='store_true', help='Skip confirmation')
import_parser.add_argument('--min-confidence', type=float, help='Minimum confidence threshold')
# Export
export_parser = subparsers.add_parser('export', help='Export instincts')
export_parser.add_argument('--output', '-o', help='Output file')
export_parser.add_argument('--domain', help='Filter by domain')
export_parser.add_argument('--min-confidence', type=float, help='Minimum confidence')
# Evolve
evolve_parser = subparsers.add_parser('evolve', help='Analyze and evolve instincts')
evolve_parser.add_argument('--generate', action='store_true', help='Generate evolved structures')
args = parser.parse_args()
if args.command == 'status':
return cmd_status(args)
elif args.command == 'import':
return cmd_import(args)
elif args.command == 'export':
return cmd_export(args)
elif args.command == 'evolve':
return cmd_evolve(args)
else:
parser.print_help()
return 1
if __name__ == '__main__':
sys.exit(main() or 0)

82
skills/continuous-learning-v2/scripts/test_parse_instinct.py Normal file
View File

@@ -0,0 +1,82 @@
"""Tests for parse_instinct_file() — verifies content after frontmatter is preserved."""
import importlib.util
import os
# Load instinct-cli.py (hyphenated filename requires importlib)
_spec = importlib.util.spec_from_file_location(
"instinct_cli",
os.path.join(os.path.dirname(__file__), "instinct-cli.py"),
)
_mod = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(_mod)
parse_instinct_file = _mod.parse_instinct_file
MULTI_SECTION = """\
---
id: instinct-a
trigger: "when coding"
confidence: 0.9
domain: general
---
## Action
Do thing A.
## Examples
- Example A1
---
id: instinct-b
trigger: "when testing"
confidence: 0.7
domain: testing
---
## Action
Do thing B.
"""
def test_multiple_instincts_preserve_content():
result = parse_instinct_file(MULTI_SECTION)
assert len(result) == 2
assert "Do thing A." in result[0]["content"]
assert "Example A1" in result[0]["content"]
assert "Do thing B." in result[1]["content"]
def test_single_instinct_preserves_content():
content = """\
---
id: solo
trigger: "when reviewing"
confidence: 0.8
domain: review
---
## Action
Check for security issues.
## Evidence
Prevents vulnerabilities.
"""
result = parse_instinct_file(content)
assert len(result) == 1
assert "Check for security issues." in result[0]["content"]
assert "Prevents vulnerabilities." in result[0]["content"]
def test_empty_content_no_error():
content = """\
---
id: empty
trigger: "placeholder"
confidence: 0.5
domain: general
---
"""
result = parse_instinct_file(content)
assert len(result) == 1
assert result[0]["content"] == ""

118
skills/continuous-learning/SKILL.md Normal file
View File

@@ -0,0 +1,118 @@
---
name: continuous-learning
description: Automatically extract reusable patterns from Claude Code sessions and save them as learned skills for future use.
---
# Continuous Learning Skill
Automatically evaluates Claude Code sessions on end to extract reusable patterns that can be saved as learned skills.
## When to Activate
- Setting up automatic pattern extraction from Claude Code sessions
- Configuring the Stop hook for session evaluation
- Reviewing or curating learned skills in `~/.claude/skills/learned/`
- Adjusting extraction thresholds or pattern categories
- Comparing v1 (this) vs v2 (instinct-based) approaches
## How It Works
This skill runs as a **Stop hook** at the end of each session:
1. **Session Evaluation**: Checks if session has enough messages (default: 10+)
2. **Pattern Detection**: Identifies extractable patterns from the session
3. **Skill Extraction**: Saves useful patterns to `~/.claude/skills/learned/`
## Configuration
Edit `config.json` to customize:
```json
{
"min_session_length": 10,
"extraction_threshold": "medium",
"auto_approve": false,
"learned_skills_path": "~/.claude/skills/learned/",
"patterns_to_detect": [
"error_resolution",
"user_corrections",
"workarounds",
"debugging_techniques",
"project_specific"
],
"ignore_patterns": [
"simple_typos",
"one_time_fixes",
"external_api_issues"
]
}
```
## Pattern Types
| Pattern | Description |
|---------|-------------|
| `error_resolution` | How specific errors were resolved |
| `user_corrections` | Patterns from user corrections |
| `workarounds` | Solutions to framework/library quirks |
| `debugging_techniques` | Effective debugging approaches |
| `project_specific` | Project-specific conventions |
## Hook Setup
Add to your `~/.claude/settings.json`:
```json
{
"hooks": {
"Stop": [{
"matcher": "*",
"hooks": [{
"type": "command",
"command": "~/.claude/skills/continuous-learning/evaluate-session.sh"
}]
}]
}
}
```
## Why Stop Hook?
- **Lightweight**: Runs once at session end
- **Non-blocking**: Doesn't add latency to every message
- **Complete context**: Has access to full session transcript
## Related
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Section on continuous learning
- `/learn` command - Manual pattern extraction mid-session
---
## Comparison Notes (Research: Jan 2025)
### vs Homunculus (github.com/humanplane/homunculus)
Homunculus v2 takes a more sophisticated approach:
| Feature | Our Approach | Homunculus v2 |
|---------|--------------|---------------|
| Observation | Stop hook (end of session) | PreToolUse/PostToolUse hooks (100% reliable) |
| Analysis | Main context | Background agent (Haiku) |
| Granularity | Full skills | Atomic "instincts" |
| Confidence | None | 0.3-0.9 weighted |
| Evolution | Direct to skill | Instincts → cluster → skill/command/agent |
| Sharing | None | Export/import instincts |
**Key insight from homunculus:**
> "v1 relied on skills to observe. Skills are probabilistic—they fire ~50-80% of the time. v2 uses hooks for observation (100% reliable) and instincts as the atomic unit of learned behavior."
### Potential v2 Enhancements
1. **Instinct-based learning** - Smaller, atomic behaviors with confidence scoring
2. **Background observer** - Haiku agent analyzing in parallel
3. **Confidence decay** - Instincts lose confidence if contradicted
4. **Domain tagging** - code-style, testing, git, debugging, etc.
5. **Evolution path** - Cluster related instincts into skills/commands
See: `/Users/affoon/Documents/tasks/12-continuous-learning-v2.md` for full spec.

18
skills/continuous-learning/config.json Normal file
View File

@@ -0,0 +1,18 @@
{
"min_session_length": 10,
"extraction_threshold": "medium",
"auto_approve": false,
"learned_skills_path": "~/.claude/skills/learned/",
"patterns_to_detect": [
"error_resolution",
"user_corrections",
"workarounds",
"debugging_techniques",
"project_specific"
],
"ignore_patterns": [
"simple_typos",
"one_time_fixes",
"external_api_issues"
]
}

69
skills/continuous-learning/evaluate-session.sh Executable file
View File

@@ -0,0 +1,69 @@
#!/bin/bash
# Continuous Learning - Session Evaluator
# Runs on Stop hook to extract reusable patterns from Claude Code sessions
#
# Why Stop hook instead of UserPromptSubmit:
# - Stop runs once at session end (lightweight)
# - UserPromptSubmit runs every message (heavy, adds latency)
#
# Hook config (in ~/.claude/settings.json):
# {
# "hooks": {
# "Stop": [{
# "matcher": "*",
# "hooks": [{
# "type": "command",
# "command": "~/.claude/skills/continuous-learning/evaluate-session.sh"
# }]
# }]
# }
# }
#
# Patterns to detect: error_resolution, debugging_techniques, workarounds, project_specific
# Patterns to ignore: simple_typos, one_time_fixes, external_api_issues
# Extracted skills saved to: ~/.claude/skills/learned/
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CONFIG_FILE="$SCRIPT_DIR/config.json"
LEARNED_SKILLS_PATH="${HOME}/.claude/skills/learned"
MIN_SESSION_LENGTH=10
# Load config if exists
if [ -f "$CONFIG_FILE" ]; then
if ! command -v jq &>/dev/null; then
echo "[ContinuousLearning] jq is required to parse config.json but not installed, using defaults" >&2
else
MIN_SESSION_LENGTH=$(jq -r '.min_session_length // 10' "$CONFIG_FILE")
LEARNED_SKILLS_PATH=$(jq -r '.learned_skills_path // "~/.claude/skills/learned/"' "$CONFIG_FILE" | sed "s|~|$HOME|")
fi
fi
# Ensure learned skills directory exists
mkdir -p "$LEARNED_SKILLS_PATH"
# Get transcript path from stdin JSON (Claude Code hook input)
# Falls back to env var for backwards compatibility
stdin_data=$(cat)
transcript_path=$(echo "$stdin_data" | grep -o '"transcript_path":"[^"]*"' | head -1 | cut -d'"' -f4)
if [ -z "$transcript_path" ]; then
transcript_path="${CLAUDE_TRANSCRIPT_PATH:-}"
fi
if [ -z "$transcript_path" ] || [ ! -f "$transcript_path" ]; then
exit 0
fi
# Count messages in session
message_count=$(grep -c '"type":"user"' "$transcript_path" 2>/dev/null || echo "0")
# Skip short sessions
if [ "$message_count" -lt "$MIN_SESSION_LENGTH" ]; then
echo "[ContinuousLearning] Session too short ($message_count messages), skipping" >&2
exit 0
fi
# Signal to Claude that session should be evaluated for extractable patterns
echo "[ContinuousLearning] Session has $message_count messages - evaluate for extractable patterns" >&2
echo "[ContinuousLearning] Save learned skills to: $LEARNED_SKILLS_PATH" >&2

322
skills/cpp-testing/SKILL.md Normal file
View File

@@ -0,0 +1,322 @@
---
name: cpp-testing
description: Use only when writing/updating/fixing C++ tests, configuring GoogleTest/CTest, diagnosing failing or flaky tests, or adding coverage/sanitizers.
---
# C++ Testing (Agent Skill)
Agent-focused testing workflow for modern C++ (C++17/20) using GoogleTest/GoogleMock with CMake/CTest.
## When to Use
- Writing new C++ tests or fixing existing tests
- Designing unit/integration test coverage for C++ components
- Adding test coverage, CI gating, or regression protection
- Configuring CMake/CTest workflows for consistent execution
- Investigating test failures or flaky behavior
- Enabling sanitizers for memory/race diagnostics
### When NOT to Use
- Implementing new product features without test changes
- Large-scale refactors unrelated to test coverage or failures
- Performance tuning without test regressions to validate
- Non-C++ projects or non-test tasks
## Core Concepts
- **TDD loop**: red → green → refactor (tests first, minimal fix, then cleanups).
- **Isolation**: prefer dependency injection and fakes over global state.
- **Test layout**: `tests/unit`, `tests/integration`, `tests/testdata`.
- **Mocks vs fakes**: mock for interactions, fake for stateful behavior.
- **CTest discovery**: use `gtest_discover_tests()` for stable test discovery.
- **CI signal**: run subset first, then full suite with `--output-on-failure`.
## TDD Workflow
Follow the RED → GREEN → REFACTOR loop:
1. **RED**: write a failing test that captures the new behavior
2. **GREEN**: implement the smallest change to pass
3. **REFACTOR**: clean up while tests stay green
```cpp
// tests/add_test.cpp
#include <gtest/gtest.h>
int Add(int a, int b); // Provided by production code.
TEST(AddTest, AddsTwoNumbers) { // RED
EXPECT_EQ(Add(2, 3), 5);
}
// src/add.cpp
int Add(int a, int b) { // GREEN
return a + b;
}
// REFACTOR: simplify/rename once tests pass
```
## Code Examples
### Basic Unit Test (gtest)
```cpp
// tests/calculator_test.cpp
#include <gtest/gtest.h>
int Add(int a, int b); // Provided by production code.
TEST(CalculatorTest, AddsTwoNumbers) {
EXPECT_EQ(Add(2, 3), 5);
}
```
### Fixture (gtest)
```cpp
// tests/user_store_test.cpp
// Pseudocode stub: replace UserStore/User with project types.
#include <gtest/gtest.h>
#include <memory>
#include <optional>
#include <string>
struct User { std::string name; };
class UserStore {
public:
explicit UserStore(std::string /*path*/) {}
void Seed(std::initializer_list<User> /*users*/) {}
std::optional<User> Find(const std::string &/*name*/) { return User{"alice"}; }
};
class UserStoreTest : public ::testing::Test {
protected:
void SetUp() override {
store = std::make_unique<UserStore>(":memory:");
store->Seed({{"alice"}, {"bob"}});
}
std::unique_ptr<UserStore> store;
};
TEST_F(UserStoreTest, FindsExistingUser) {
auto user = store->Find("alice");
ASSERT_TRUE(user.has_value());
EXPECT_EQ(user->name, "alice");
}
```
### Mock (gmock)
```cpp
// tests/notifier_test.cpp
#include <gmock/gmock.h>
#include <gtest/gtest.h>
#include <string>
class Notifier {
public:
virtual ~Notifier() = default;
virtual void Send(const std::string &message) = 0;
};
class MockNotifier : public Notifier {
public:
MOCK_METHOD(void, Send, (const std::string &message), (override));
};
class Service {
public:
explicit Service(Notifier &notifier) : notifier_(notifier) {}
void Publish(const std::string &message) { notifier_.Send(message); }
private:
Notifier &notifier_;
};
TEST(ServiceTest, SendsNotifications) {
MockNotifier notifier;
Service service(notifier);
EXPECT_CALL(notifier, Send("hello")).Times(1);
service.Publish("hello");
}
```
### CMake/CTest Quickstart
```cmake
# CMakeLists.txt (excerpt)
cmake_minimum_required(VERSION 3.20)
project(example LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
include(FetchContent)
# Prefer project-locked versions. If using a tag, use a pinned version per project policy.
set(GTEST_VERSION v1.17.0) # Adjust to project policy.
FetchContent_Declare(
googletest
URL https://github.com/google/googletest/archive/refs/tags/${GTEST_VERSION}.zip
)
FetchContent_MakeAvailable(googletest)
add_executable(example_tests
tests/calculator_test.cpp
src/calculator.cpp
)
target_link_libraries(example_tests GTest::gtest GTest::gmock GTest::gtest_main)
enable_testing()
include(GoogleTest)
gtest_discover_tests(example_tests)
```
```bash
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j
ctest --test-dir build --output-on-failure
```
## Running Tests
```bash
ctest --test-dir build --output-on-failure
ctest --test-dir build -R ClampTest
ctest --test-dir build -R "UserStoreTest.*" --output-on-failure
```
```bash
./build/example_tests --gtest_filter=ClampTest.*
./build/example_tests --gtest_filter=UserStoreTest.FindsExistingUser
```
## Debugging Failures
1. Re-run the single failing test with gtest filter.
2. Add scoped logging around the failing assertion.
3. Re-run with sanitizers enabled.
4. Expand to full suite once the root cause is fixed.
## Coverage
Prefer target-level settings instead of global flags.
```cmake
option(ENABLE_COVERAGE "Enable coverage flags" OFF)
if(ENABLE_COVERAGE)
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
target_compile_options(example_tests PRIVATE --coverage)
target_link_options(example_tests PRIVATE --coverage)
elseif(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
target_compile_options(example_tests PRIVATE -fprofile-instr-generate -fcoverage-mapping)
target_link_options(example_tests PRIVATE -fprofile-instr-generate)
endif()
endif()
```
GCC + gcov + lcov:
```bash
cmake -S . -B build-cov -DENABLE_COVERAGE=ON
cmake --build build-cov -j
ctest --test-dir build-cov
lcov --capture --directory build-cov --output-file coverage.info
lcov --remove coverage.info '/usr/*' --output-file coverage.info
genhtml coverage.info --output-directory coverage
```
Clang + llvm-cov:
```bash
cmake -S . -B build-llvm -DENABLE_COVERAGE=ON -DCMAKE_CXX_COMPILER=clang++
cmake --build build-llvm -j
LLVM_PROFILE_FILE="build-llvm/default.profraw" ctest --test-dir build-llvm
llvm-profdata merge -sparse build-llvm/default.profraw -o build-llvm/default.profdata
llvm-cov report build-llvm/example_tests -instr-profile=build-llvm/default.profdata
```
## Sanitizers
```cmake
option(ENABLE_ASAN "Enable AddressSanitizer" OFF)
option(ENABLE_UBSAN "Enable UndefinedBehaviorSanitizer" OFF)
option(ENABLE_TSAN "Enable ThreadSanitizer" OFF)
if(ENABLE_ASAN)
add_compile_options(-fsanitize=address -fno-omit-frame-pointer)
add_link_options(-fsanitize=address)
endif()
if(ENABLE_UBSAN)
add_compile_options(-fsanitize=undefined -fno-omit-frame-pointer)
add_link_options(-fsanitize=undefined)
endif()
if(ENABLE_TSAN)
add_compile_options(-fsanitize=thread)
add_link_options(-fsanitize=thread)
endif()
```
## Flaky Tests Guardrails
- Never use `sleep` for synchronization; use condition variables or latches.
- Make temp directories unique per test and always clean them.
- Avoid real time, network, or filesystem dependencies in unit tests.
- Use deterministic seeds for randomized inputs.
## Best Practices
### DO
- Keep tests deterministic and isolated
- Prefer dependency injection over globals
- Use `ASSERT_*` for preconditions, `EXPECT_*` for multiple checks
- Separate unit vs integration tests in CTest labels or directories
- Run sanitizers in CI for memory and race detection
### DON'T
- Don't depend on real time or network in unit tests
- Don't use sleeps as synchronization when a condition variable can be used
- Don't over-mock simple value objects
- Don't use brittle string matching for non-critical logs
### Common Pitfalls
- **Using fixed temp paths** → Generate unique temp directories per test and clean them.
- **Relying on wall clock time** → Inject a clock or use fake time sources.
- **Flaky concurrency tests** → Use condition variables/latches and bounded waits.
- **Hidden global state** → Reset global state in fixtures or remove globals.
- **Over-mocking** → Prefer fakes for stateful behavior and only mock interactions.
- **Missing sanitizer runs** → Add ASan/UBSan/TSan builds in CI.
- **Coverage on debug-only builds** → Ensure coverage targets use consistent flags.
## Optional Appendix: Fuzzing / Property Testing
Only use if the project already supports LLVM/libFuzzer or a property-testing library.
- **libFuzzer**: best for pure functions with minimal I/O.
- **RapidCheck**: property-based tests to validate invariants.
Minimal libFuzzer harness (pseudocode: replace ParseConfig):
```cpp
#include <cstddef>
#include <cstdint>
#include <string>
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
std::string input(reinterpret_cast<const char *>(data), size);
// ParseConfig(input); // project function
return 0;
}
```
## Alternatives to GoogleTest
- **Catch2**: header-only, expressive matchers
- **doctest**: lightweight, minimal compile overhead

334
skills/database-migrations/SKILL.md Normal file
View File

@@ -0,0 +1,334 @@
---
name: database-migrations
description: Database migration best practices for schema changes, data migrations, rollbacks, and zero-downtime deployments across PostgreSQL, MySQL, and common ORMs (Prisma, Drizzle, Django, TypeORM, golang-migrate).
---
# Database Migration Patterns
Safe, reversible database schema changes for production systems.
## When to Activate
- Creating or altering database tables
- Adding/removing columns or indexes
- Running data migrations (backfill, transform)
- Planning zero-downtime schema changes
- Setting up migration tooling for a new project
## Core Principles
1. **Every change is a migration** — never alter production databases manually
2. **Migrations are forward-only in production** — rollbacks use new forward migrations
3. **Schema and data migrations are separate** — never mix DDL and DML in one migration
4. **Test migrations against production-sized data** — a migration that works on 100 rows may lock on 10M
5. **Migrations are immutable once deployed** — never edit a migration that has run in production
## Migration Safety Checklist
Before applying any migration:
- [ ] Migration has both UP and DOWN (or is explicitly marked irreversible)
- [ ] No full table locks on large tables (use concurrent operations)
- [ ] New columns have defaults or are nullable (never add NOT NULL without default)
- [ ] Indexes created concurrently (not inline with CREATE TABLE for existing tables)
- [ ] Data backfill is a separate migration from schema change
- [ ] Tested against a copy of production data
- [ ] Rollback plan documented
## PostgreSQL Patterns
### Adding a Column Safely
```sql
-- GOOD: Nullable column, no lock
ALTER TABLE users ADD COLUMN avatar_url TEXT;
-- GOOD: Column with default (Postgres 11+ is instant, no rewrite)
ALTER TABLE users ADD COLUMN is_active BOOLEAN NOT NULL DEFAULT true;
-- BAD: NOT NULL without default on existing table (requires full rewrite)
ALTER TABLE users ADD COLUMN role TEXT NOT NULL;
-- This locks the table and rewrites every row
```
### Adding an Index Without Downtime
```sql
-- BAD: Blocks writes on large tables
CREATE INDEX idx_users_email ON users (email);
-- GOOD: Non-blocking, allows concurrent writes
CREATE INDEX CONCURRENTLY idx_users_email ON users (email);
-- Note: CONCURRENTLY cannot run inside a transaction block
-- Most migration tools need special handling for this
```
### Renaming a Column (Zero-Downtime)
Never rename directly in production. Use the expand-contract pattern:
```sql
-- Step 1: Add new column (migration 001)
ALTER TABLE users ADD COLUMN display_name TEXT;
-- Step 2: Backfill data (migration 002, data migration)
UPDATE users SET display_name = username WHERE display_name IS NULL;
-- Step 3: Update application code to read/write both columns
-- Deploy application changes
-- Step 4: Stop writing to old column, drop it (migration 003)
ALTER TABLE users DROP COLUMN username;
```
### Removing a Column Safely
```sql
-- Step 1: Remove all application references to the column
-- Step 2: Deploy application without the column reference
-- Step 3: Drop column in next migration
ALTER TABLE orders DROP COLUMN legacy_status;
-- For Django: use SeparateDatabaseAndState to remove from model
-- without generating DROP COLUMN (then drop in next migration)
```
### Large Data Migrations
```sql
-- BAD: Updates all rows in one transaction (locks table)
UPDATE users SET normalized_email = LOWER(email);
-- GOOD: Batch update with progress
DO $$
DECLARE
batch_size INT := 10000;
rows_updated INT;
BEGIN
LOOP
UPDATE users
SET normalized_email = LOWER(email)
WHERE id IN (
SELECT id FROM users
WHERE normalized_email IS NULL
LIMIT batch_size
FOR UPDATE SKIP LOCKED
);
GET DIAGNOSTICS rows_updated = ROW_COUNT;
RAISE NOTICE 'Updated % rows', rows_updated;
EXIT WHEN rows_updated = 0;
COMMIT;
END LOOP;
END $$;
```
## Prisma (TypeScript/Node.js)
### Workflow
```bash
# Create migration from schema changes
npx prisma migrate dev --name add_user_avatar
# Apply pending migrations in production
npx prisma migrate deploy
# Reset database (dev only)
npx prisma migrate reset
# Generate client after schema changes
npx prisma generate
```
### Schema Example
```prisma
model User {
id String @id @default(cuid())
email String @unique
name String?
avatarUrl String? @map("avatar_url")
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
orders Order[]
@@map("users")
@@index([email])
}
```
### Custom SQL Migration
For operations Prisma cannot express (concurrent indexes, data backfills):
```bash
# Create empty migration, then edit the SQL manually
npx prisma migrate dev --create-only --name add_email_index
```
```sql
-- migrations/20240115_add_email_index/migration.sql
-- Prisma cannot generate CONCURRENTLY, so we write it manually
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_users_email ON users (email);
```
## Drizzle (TypeScript/Node.js)
### Workflow
```bash
# Generate migration from schema changes
npx drizzle-kit generate
# Apply migrations
npx drizzle-kit migrate
# Push schema directly (dev only, no migration file)
npx drizzle-kit push
```
### Schema Example
```typescript
import { pgTable, text, timestamp, uuid, boolean } from "drizzle-orm/pg-core";
export const users = pgTable("users", {
id: uuid("id").primaryKey().defaultRandom(),
email: text("email").notNull().unique(),
name: text("name"),
isActive: boolean("is_active").notNull().default(true),
createdAt: timestamp("created_at").notNull().defaultNow(),
updatedAt: timestamp("updated_at").notNull().defaultNow(),
});
```
## Django (Python)
### Workflow
```bash
# Generate migration from model changes
python manage.py makemigrations
# Apply migrations
python manage.py migrate
# Show migration status
python manage.py showmigrations
# Generate empty migration for custom SQL
python manage.py makemigrations --empty app_name -n description
```
### Data Migration
```python
from django.db import migrations
def backfill_display_names(apps, schema_editor):
User = apps.get_model("accounts", "User")
batch_size = 5000
users = User.objects.filter(display_name="")
while users.exists():
batch = list(users[:batch_size])
for user in batch:
user.display_name = user.username
User.objects.bulk_update(batch, ["display_name"], batch_size=batch_size)
def reverse_backfill(apps, schema_editor):
pass # Data migration, no reverse needed
class Migration(migrations.Migration):
dependencies = [("accounts", "0015_add_display_name")]
operations = [
migrations.RunPython(backfill_display_names, reverse_backfill),
]
```
### SeparateDatabaseAndState
Remove a column from the Django model without dropping it from the database immediately:
```python
class Migration(migrations.Migration):
operations = [
migrations.SeparateDatabaseAndState(
state_operations=[
migrations.RemoveField(model_name="user", name="legacy_field"),
],
database_operations=[], # Don't touch the DB yet
),
]
```
## golang-migrate (Go)
### Workflow
```bash
# Create migration pair
migrate create -ext sql -dir migrations -seq add_user_avatar
# Apply all pending migrations
migrate -path migrations -database "$DATABASE_URL" up
# Rollback last migration
migrate -path migrations -database "$DATABASE_URL" down 1
# Force version (fix dirty state)
migrate -path migrations -database "$DATABASE_URL" force VERSION
```
### Migration Files
```sql
-- migrations/000003_add_user_avatar.up.sql
ALTER TABLE users ADD COLUMN avatar_url TEXT;
CREATE INDEX CONCURRENTLY idx_users_avatar ON users (avatar_url) WHERE avatar_url IS NOT NULL;
-- migrations/000003_add_user_avatar.down.sql
DROP INDEX IF EXISTS idx_users_avatar;
ALTER TABLE users DROP COLUMN IF EXISTS avatar_url;
```
## Zero-Downtime Migration Strategy
For critical production changes, follow the expand-contract pattern:
```
Phase 1: EXPAND
- Add new column/table (nullable or with default)
- Deploy: app writes to BOTH old and new
- Backfill existing data
Phase 2: MIGRATE
- Deploy: app reads from NEW, writes to BOTH
- Verify data consistency
Phase 3: CONTRACT
- Deploy: app only uses NEW
- Drop old column/table in separate migration
```
### Timeline Example
```
Day 1: Migration adds new_status column (nullable)
Day 1: Deploy app v2 — writes to both status and new_status
Day 2: Run backfill migration for existing rows
Day 3: Deploy app v3 — reads from new_status only
Day 7: Migration drops old status column
```
## Anti-Patterns
| Anti-Pattern | Why It Fails | Better Approach |
|-------------|-------------|-----------------|
| Manual SQL in production | No audit trail, unrepeatable | Always use migration files |
| Editing deployed migrations | Causes drift between environments | Create new migration instead |
| NOT NULL without default | Locks table, rewrites all rows | Add nullable, backfill, then add constraint |
| Inline index on large table | Blocks writes during build | CREATE INDEX CONCURRENTLY |
| Schema + data in one migration | Hard to rollback, long transactions | Separate migrations |
| Dropping column before removing code | Application errors on missing column | Remove code first, drop column next deploy |

71
skills/database.md Normal file
View File

@@ -0,0 +1,71 @@
# Database & SQL Skill
## Installed Tools
- PostgreSQL (docker)
- MongoDB (docker)
## Docker Compose for Databases
### PostgreSQL
```yaml
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: user
POSTGRES_PASSWORD: password
POSTGRES_DB: mydb
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
```
### MongoDB
```yaml
services:
mongodb:
image: mongo:7
environment:
MONGO_INITDB_ROOT_USERNAME: user
MONGO_INITDB_ROOT_PASSWORD: password
volumes:
- mongo_data:/data/db
ports:
- "27017:27017"
```
### PostgreSQL + pgAdmin
```yaml
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: admin
POSTGRES_PASSWORD: secret
POSTGRES_DB: mydb
ports:
- "5432:5432"
pgadmin:
image: dpage/pgadmin4
environment:
PGADMIN_DEFAULT_EMAIL: admin@test.com
PGADMIN_DEFAULT_PASSWORD: admin
ports:
- "8080:80"
```
## Common Commands
```bash
# PostgreSQL
docker exec -it container psql -U user -d mydb
# MongoDB
docker exec -it container mongosh -u user -p password
```
## GUI Tools
- DBeaver (multi-database)
- pgAdmin (PostgreSQL)
- MongoDB Compass (MongoDB)

426
skills/deployment-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,426 @@
---
name: deployment-patterns
description: Deployment workflows, CI/CD pipeline patterns, Docker containerization, health checks, rollback strategies, and production readiness checklists for web applications.
---
# Deployment Patterns
Production deployment workflows and CI/CD best practices.
## When to Activate
- Setting up CI/CD pipelines
- Dockerizing an application
- Planning deployment strategy (blue-green, canary, rolling)
- Implementing health checks and readiness probes
- Preparing for a production release
- Configuring environment-specific settings
## Deployment Strategies
### Rolling Deployment (Default)
Replace instances gradually — old and new versions run simultaneously during rollout.
```
Instance 1: v1 → v2 (update first)
Instance 2: v1 (still running v1)
Instance 3: v1 (still running v1)
Instance 1: v2
Instance 2: v1 → v2 (update second)
Instance 3: v1
Instance 1: v2
Instance 2: v2
Instance 3: v1 → v2 (update last)
```
**Pros:** Zero downtime, gradual rollout
**Cons:** Two versions run simultaneously — requires backward-compatible changes
**Use when:** Standard deployments, backward-compatible changes
### Blue-Green Deployment
Run two identical environments. Switch traffic atomically.
```
Blue (v1) ← traffic
Green (v2) idle, running new version
# After verification:
Blue (v1) idle (becomes standby)
Green (v2) ← traffic
```
**Pros:** Instant rollback (switch back to blue), clean cutover
**Cons:** Requires 2x infrastructure during deployment
**Use when:** Critical services, zero-tolerance for issues
### Canary Deployment
Route a small percentage of traffic to the new version first.
```
v1: 95% of traffic
v2: 5% of traffic (canary)
# If metrics look good:
v1: 50% of traffic
v2: 50% of traffic
# Final:
v2: 100% of traffic
```
**Pros:** Catches issues with real traffic before full rollout
**Cons:** Requires traffic splitting infrastructure, monitoring
**Use when:** High-traffic services, risky changes, feature flags
## Docker
### Multi-Stage Dockerfile (Node.js)
```dockerfile
# Stage 1: Install dependencies
FROM node:22-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --production=false
# Stage 2: Build
FROM node:22-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
RUN npm prune --production
# Stage 3: Production image
FROM node:22-alpine AS runner
WORKDIR /app
RUN addgroup -g 1001 -S appgroup && adduser -S appuser -u 1001
USER appuser
COPY --from=builder --chown=appuser:appgroup /app/node_modules ./node_modules
COPY --from=builder --chown=appuser:appgroup /app/dist ./dist
COPY --from=builder --chown=appuser:appgroup /app/package.json ./
ENV NODE_ENV=production
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]
```
### Multi-Stage Dockerfile (Go)
```dockerfile
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /server ./cmd/server
FROM alpine:3.19 AS runner
RUN apk --no-cache add ca-certificates
RUN adduser -D -u 1001 appuser
USER appuser
COPY --from=builder /server /server
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s CMD wget -qO- http://localhost:8080/health || exit 1
CMD ["/server"]
```
### Multi-Stage Dockerfile (Python/Django)
```dockerfile
FROM python:3.12-slim AS builder
WORKDIR /app
RUN pip install --no-cache-dir uv
COPY requirements.txt .
RUN uv pip install --system --no-cache -r requirements.txt
FROM python:3.12-slim AS runner
WORKDIR /app
RUN useradd -r -u 1001 appuser
USER appuser
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY . .
ENV PYTHONUNBUFFERED=1
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health/')" || exit 1
CMD ["gunicorn", "config.wsgi:application", "--bind", "0.0.0.0:8000", "--workers", "4"]
```
### Docker Best Practices
```
# GOOD practices
- Use specific version tags (node:22-alpine, not node:latest)
- Multi-stage builds to minimize image size
- Run as non-root user
- Copy dependency files first (layer caching)
- Use .dockerignore to exclude node_modules, .git, tests
- Add HEALTHCHECK instruction
- Set resource limits in docker-compose or k8s
# BAD practices
- Running as root
- Using :latest tags
- Copying entire repo in one COPY layer
- Installing dev dependencies in production image
- Storing secrets in image (use env vars or secrets manager)
```
## CI/CD Pipeline
### GitHub Actions (Standard Pipeline)
```yaml
name: CI/CD
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- run: npm run lint
- run: npm run typecheck
- run: npm test -- --coverage
- uses: actions/upload-artifact@v4
if: always()
with:
name: coverage
path: coverage/
build:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/build-push-action@v5
with:
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy:
needs: build
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
environment: production
steps:
- name: Deploy to production
run: |
# Platform-specific deployment command
# Railway: railway up
# Vercel: vercel --prod
# K8s: kubectl set image deployment/app app=ghcr.io/${{ github.repository }}:${{ github.sha }}
echo "Deploying ${{ github.sha }}"
```
### Pipeline Stages
```
PR opened:
lint → typecheck → unit tests → integration tests → preview deploy
Merged to main:
lint → typecheck → unit tests → integration tests → build image → deploy staging → smoke tests → deploy production
```
## Health Checks
### Health Check Endpoint
```typescript
// Simple health check
app.get("/health", (req, res) => {
res.status(200).json({ status: "ok" });
});
// Detailed health check (for internal monitoring)
app.get("/health/detailed", async (req, res) => {
const checks = {
database: await checkDatabase(),
redis: await checkRedis(),
externalApi: await checkExternalApi(),
};
const allHealthy = Object.values(checks).every(c => c.status === "ok");
res.status(allHealthy ? 200 : 503).json({
status: allHealthy ? "ok" : "degraded",
timestamp: new Date().toISOString(),
version: process.env.APP_VERSION || "unknown",
uptime: process.uptime(),
checks,
});
});
async function checkDatabase(): Promise<HealthCheck> {
try {
await db.query("SELECT 1");
return { status: "ok", latency_ms: 2 };
} catch (err) {
return { status: "error", message: "Database unreachable" };
}
}
```
### Kubernetes Probes
```yaml
livenessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 10
periodSeconds: 30
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 2
startupProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 30 # 30 * 5s = 150s max startup time
```
## Environment Configuration
### Twelve-Factor App Pattern
```bash
# All config via environment variables — never in code
DATABASE_URL=postgres://user:pass@host:5432/db
REDIS_URL=redis://host:6379/0
API_KEY=${API_KEY} # injected by secrets manager
LOG_LEVEL=info
PORT=3000
# Environment-specific behavior
NODE_ENV=production # or staging, development
APP_ENV=production # explicit app environment
```
### Configuration Validation
```typescript
import { z } from "zod";
const envSchema = z.object({
NODE_ENV: z.enum(["development", "staging", "production"]),
PORT: z.coerce.number().default(3000),
DATABASE_URL: z.string().url(),
REDIS_URL: z.string().url(),
JWT_SECRET: z.string().min(32),
LOG_LEVEL: z.enum(["debug", "info", "warn", "error"]).default("info"),
});
// Validate at startup — fail fast if config is wrong
export const env = envSchema.parse(process.env);
```
## Rollback Strategy
### Instant Rollback
```bash
# Docker/Kubernetes: point to previous image
kubectl rollout undo deployment/app
# Vercel: promote previous deployment
vercel rollback
# Railway: redeploy previous commit
railway up --commit <previous-sha>
# Database: rollback migration (if reversible)
npx prisma migrate resolve --rolled-back <migration-name>
```
### Rollback Checklist
- [ ] Previous image/artifact is available and tagged
- [ ] Database migrations are backward-compatible (no destructive changes)
- [ ] Feature flags can disable new features without deploy
- [ ] Monitoring alerts configured for error rate spikes
- [ ] Rollback tested in staging before production release
## Production Readiness Checklist
Before any production deployment:
### Application
- [ ] All tests pass (unit, integration, E2E)
- [ ] No hardcoded secrets in code or config files
- [ ] Error handling covers all edge cases
- [ ] Logging is structured (JSON) and does not contain PII
- [ ] Health check endpoint returns meaningful status
### Infrastructure
- [ ] Docker image builds reproducibly (pinned versions)
- [ ] Environment variables documented and validated at startup
- [ ] Resource limits set (CPU, memory)
- [ ] Horizontal scaling configured (min/max instances)
- [ ] SSL/TLS enabled on all endpoints
### Monitoring
- [ ] Application metrics exported (request rate, latency, errors)
- [ ] Alerts configured for error rate > threshold
- [ ] Log aggregation set up (structured logs, searchable)
- [ ] Uptime monitoring on health endpoint
### Security
- [ ] Dependencies scanned for CVEs
- [ ] CORS configured for allowed origins only
- [ ] Rate limiting enabled on public endpoints
- [ ] Authentication and authorization verified
- [ ] Security headers set (CSP, HSTS, X-Frame-Options)
### Operations
- [ ] Rollback plan documented and tested
- [ ] Database migration tested against production-sized data
- [ ] Runbook for common failure scenarios
- [ ] On-call rotation and escalation path defined

733
skills/django-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,733 @@
---
name: django-patterns
description: Django architecture patterns, REST API design with DRF, ORM best practices, caching, signals, middleware, and production-grade Django apps.
---
# Django Development Patterns
Production-grade Django architecture patterns for scalable, maintainable applications.
## When to Activate
- Building Django web applications
- Designing Django REST Framework APIs
- Working with Django ORM and models
- Setting up Django project structure
- Implementing caching, signals, middleware
## Project Structure
### Recommended Layout
```
myproject/
├── config/
│ ├── __init__.py
│ ├── settings/
│ │ ├── __init__.py
│ │ ├── base.py # Base settings
│ │ ├── development.py # Dev settings
│ │ ├── production.py # Production settings
│ │ └── test.py # Test settings
│ ├── urls.py
│ ├── wsgi.py
│ └── asgi.py
├── manage.py
└── apps/
├── __init__.py
├── users/
│ ├── __init__.py
│ ├── models.py
│ ├── views.py
│ ├── serializers.py
│ ├── urls.py
│ ├── permissions.py
│ ├── filters.py
│ ├── services.py
│ └── tests/
└── products/
└── ...
```
### Split Settings Pattern
```python
# config/settings/base.py
from pathlib import Path
BASE_DIR = Path(__file__).resolve().parent.parent.parent
SECRET_KEY = env('DJANGO_SECRET_KEY')
DEBUG = False
ALLOWED_HOSTS = []
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'rest_framework.authtoken',
'corsheaders',
# Local apps
'apps.users',
'apps.products',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'config.urls'
WSGI_APPLICATION = 'config.wsgi.application'
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': env('DB_NAME'),
'USER': env('DB_USER'),
'PASSWORD': env('DB_PASSWORD'),
'HOST': env('DB_HOST'),
'PORT': env('DB_PORT', default='5432'),
}
}
# config/settings/development.py
from .base import *
DEBUG = True
ALLOWED_HOSTS = ['localhost', '127.0.0.1']
DATABASES['default']['NAME'] = 'myproject_dev'
INSTALLED_APPS += ['debug_toolbar']
MIDDLEWARE += ['debug_toolbar.middleware.DebugToolbarMiddleware']
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
# config/settings/production.py
from .base import *
DEBUG = False
ALLOWED_HOSTS = env.list('ALLOWED_HOSTS')
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
# Logging
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'WARNING',
'class': 'logging.FileHandler',
'filename': '/var/log/django/django.log',
},
},
'loggers': {
'django': {
'handlers': ['file'],
'level': 'WARNING',
'propagate': True,
},
},
}
```
## Model Design Patterns
### Model Best Practices
```python
from django.db import models
from django.contrib.auth.models import AbstractUser
from django.core.validators import MinValueValidator, MaxValueValidator
class User(AbstractUser):
"""Custom user model extending AbstractUser."""
email = models.EmailField(unique=True)
phone = models.CharField(max_length=20, blank=True)
birth_date = models.DateField(null=True, blank=True)
USERNAME_FIELD = 'email'
REQUIRED_FIELDS = ['username']
class Meta:
db_table = 'users'
verbose_name = 'user'
verbose_name_plural = 'users'
ordering = ['-date_joined']
def __str__(self):
return self.email
def get_full_name(self):
return f"{self.first_name} {self.last_name}".strip()
class Product(models.Model):
"""Product model with proper field configuration."""
name = models.CharField(max_length=200)
slug = models.SlugField(unique=True, max_length=250)
description = models.TextField(blank=True)
price = models.DecimalField(
max_digits=10,
decimal_places=2,
validators=[MinValueValidator(0)]
)
stock = models.PositiveIntegerField(default=0)
is_active = models.BooleanField(default=True)
category = models.ForeignKey(
'Category',
on_delete=models.CASCADE,
related_name='products'
)
tags = models.ManyToManyField('Tag', blank=True, related_name='products')
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
db_table = 'products'
ordering = ['-created_at']
indexes = [
models.Index(fields=['slug']),
models.Index(fields=['-created_at']),
models.Index(fields=['category', 'is_active']),
]
constraints = [
models.CheckConstraint(
check=models.Q(price__gte=0),
name='price_non_negative'
)
]
def __str__(self):
return self.name
def save(self, *args, **kwargs):
if not self.slug:
self.slug = slugify(self.name)
super().save(*args, **kwargs)
```
### QuerySet Best Practices
```python
from django.db import models
class ProductQuerySet(models.QuerySet):
"""Custom QuerySet for Product model."""
def active(self):
"""Return only active products."""
return self.filter(is_active=True)
def with_category(self):
"""Select related category to avoid N+1 queries."""
return self.select_related('category')
def with_tags(self):
"""Prefetch tags for many-to-many relationship."""
return self.prefetch_related('tags')
def in_stock(self):
"""Return products with stock > 0."""
return self.filter(stock__gt=0)
def search(self, query):
"""Search products by name or description."""
return self.filter(
models.Q(name__icontains=query) |
models.Q(description__icontains=query)
)
class Product(models.Model):
# ... fields ...
objects = ProductQuerySet.as_manager() # Use custom QuerySet
# Usage
Product.objects.active().with_category().in_stock()
```
### Manager Methods
```python
class ProductManager(models.Manager):
"""Custom manager for complex queries."""
def get_or_none(self, **kwargs):
"""Return object or None instead of DoesNotExist."""
try:
return self.get(**kwargs)
except self.model.DoesNotExist:
return None
def create_with_tags(self, name, price, tag_names):
"""Create product with associated tags."""
product = self.create(name=name, price=price)
tags = [Tag.objects.get_or_create(name=name)[0] for name in tag_names]
product.tags.set(tags)
return product
def bulk_update_stock(self, product_ids, quantity):
"""Bulk update stock for multiple products."""
return self.filter(id__in=product_ids).update(stock=quantity)
# In model
class Product(models.Model):
# ... fields ...
custom = ProductManager()
```
## Django REST Framework Patterns
### Serializer Patterns
```python
from rest_framework import serializers
from django.contrib.auth.password_validation import validate_password
from .models import Product, User
class ProductSerializer(serializers.ModelSerializer):
"""Serializer for Product model."""
category_name = serializers.CharField(source='category.name', read_only=True)
average_rating = serializers.FloatField(read_only=True)
discount_price = serializers.SerializerMethodField()
class Meta:
model = Product
fields = [
'id', 'name', 'slug', 'description', 'price',
'discount_price', 'stock', 'category_name',
'average_rating', 'created_at'
]
read_only_fields = ['id', 'slug', 'created_at']
def get_discount_price(self, obj):
"""Calculate discount price if applicable."""
if hasattr(obj, 'discount') and obj.discount:
return obj.price * (1 - obj.discount.percent / 100)
return obj.price
def validate_price(self, value):
"""Ensure price is non-negative."""
if value < 0:
raise serializers.ValidationError("Price cannot be negative.")
return value
class ProductCreateSerializer(serializers.ModelSerializer):
"""Serializer for creating products."""
class Meta:
model = Product
fields = ['name', 'description', 'price', 'stock', 'category']
def validate(self, data):
"""Custom validation for multiple fields."""
if data['price'] > 10000 and data['stock'] > 100:
raise serializers.ValidationError(
"Cannot have high-value products with large stock."
)
return data
class UserRegistrationSerializer(serializers.ModelSerializer):
"""Serializer for user registration."""
password = serializers.CharField(
write_only=True,
required=True,
validators=[validate_password],
style={'input_type': 'password'}
)
password_confirm = serializers.CharField(write_only=True, style={'input_type': 'password'})
class Meta:
model = User
fields = ['email', 'username', 'password', 'password_confirm']
def validate(self, data):
"""Validate passwords match."""
if data['password'] != data['password_confirm']:
raise serializers.ValidationError({
"password_confirm": "Password fields didn't match."
})
return data
def create(self, validated_data):
"""Create user with hashed password."""
validated_data.pop('password_confirm')
password = validated_data.pop('password')
user = User.objects.create(**validated_data)
user.set_password(password)
user.save()
return user
```
### ViewSet Patterns
```python
from rest_framework import viewsets, status, filters
from rest_framework.decorators import action
from rest_framework.response import Response
from rest_framework.permissions import IsAuthenticated, IsAdminUser
from django_filters.rest_framework import DjangoFilterBackend
from .models import Product
from .serializers import ProductSerializer, ProductCreateSerializer
from .permissions import IsOwnerOrReadOnly
from .filters import ProductFilter
from .services import ProductService
class ProductViewSet(viewsets.ModelViewSet):
"""ViewSet for Product model."""
queryset = Product.objects.select_related('category').prefetch_related('tags')
permission_classes = [IsAuthenticated, IsOwnerOrReadOnly]
filter_backends = [DjangoFilterBackend, filters.SearchFilter, filters.OrderingFilter]
filterset_class = ProductFilter
search_fields = ['name', 'description']
ordering_fields = ['price', 'created_at', 'name']
ordering = ['-created_at']
def get_serializer_class(self):
"""Return appropriate serializer based on action."""
if self.action == 'create':
return ProductCreateSerializer
return ProductSerializer
def perform_create(self, serializer):
"""Save with user context."""
serializer.save(created_by=self.request.user)
@action(detail=False, methods=['get'])
def featured(self, request):
"""Return featured products."""
featured = self.queryset.filter(is_featured=True)[:10]
serializer = self.get_serializer(featured, many=True)
return Response(serializer.data)
@action(detail=True, methods=['post'])
def purchase(self, request, pk=None):
"""Purchase a product."""
product = self.get_object()
service = ProductService()
result = service.purchase(product, request.user)
return Response(result, status=status.HTTP_201_CREATED)
@action(detail=False, methods=['get'], permission_classes=[IsAuthenticated])
def my_products(self, request):
"""Return products created by current user."""
products = self.queryset.filter(created_by=request.user)
page = self.paginate_queryset(products)
serializer = self.get_serializer(page, many=True)
return self.get_paginated_response(serializer.data)
```
### Custom Actions
```python
from rest_framework.decorators import api_view, permission_classes
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
@api_view(['POST'])
@permission_classes([IsAuthenticated])
def add_to_cart(request):
"""Add product to user cart."""
product_id = request.data.get('product_id')
quantity = request.data.get('quantity', 1)
try:
product = Product.objects.get(id=product_id)
except Product.DoesNotExist:
return Response(
{'error': 'Product not found'},
status=status.HTTP_404_NOT_FOUND
)
cart, _ = Cart.objects.get_or_create(user=request.user)
CartItem.objects.create(
cart=cart,
product=product,
quantity=quantity
)
return Response({'message': 'Added to cart'}, status=status.HTTP_201_CREATED)
```
## Service Layer Pattern
```python
# apps/orders/services.py
from typing import Optional
from django.db import transaction
from .models import Order, OrderItem
class OrderService:
"""Service layer for order-related business logic."""
@staticmethod
@transaction.atomic
def create_order(user, cart: Cart) -> Order:
"""Create order from cart."""
order = Order.objects.create(
user=user,
total_price=cart.total_price
)
for item in cart.items.all():
OrderItem.objects.create(
order=order,
product=item.product,
quantity=item.quantity,
price=item.product.price
)
# Clear cart
cart.items.all().delete()
return order
@staticmethod
def process_payment(order: Order, payment_data: dict) -> bool:
"""Process payment for order."""
# Integration with payment gateway
payment = PaymentGateway.charge(
amount=order.total_price,
token=payment_data['token']
)
if payment.success:
order.status = Order.Status.PAID
order.save()
# Send confirmation email
OrderService.send_confirmation_email(order)
return True
return False
@staticmethod
def send_confirmation_email(order: Order):
"""Send order confirmation email."""
# Email sending logic
pass
```
## Caching Strategies
### View-Level Caching
```python
from django.views.decorators.cache import cache_page
from django.utils.decorators import method_decorator
@method_decorator(cache_page(60 * 15), name='dispatch') # 15 minutes
class ProductListView(generic.ListView):
model = Product
template_name = 'products/list.html'
context_object_name = 'products'
```
### Template Fragment Caching
```django
{% load cache %}
{% cache 500 sidebar %}
... expensive sidebar content ...
{% endcache %}
```
### Low-Level Caching
```python
from django.core.cache import cache
def get_featured_products():
"""Get featured products with caching."""
cache_key = 'featured_products'
products = cache.get(cache_key)
if products is None:
products = list(Product.objects.filter(is_featured=True))
cache.set(cache_key, products, timeout=60 * 15) # 15 minutes
return products
```
### QuerySet Caching
```python
from django.core.cache import cache
def get_popular_categories():
cache_key = 'popular_categories'
categories = cache.get(cache_key)
if categories is None:
categories = list(Category.objects.annotate(
product_count=Count('products')
).filter(product_count__gt=10).order_by('-product_count')[:20])
cache.set(cache_key, categories, timeout=60 * 60) # 1 hour
return categories
```
## Signals
### Signal Patterns
```python
# apps/users/signals.py
from django.db.models.signals import post_save
from django.dispatch import receiver
from django.contrib.auth import get_user_model
from .models import Profile
User = get_user_model()
@receiver(post_save, sender=User)
def create_user_profile(sender, instance, created, **kwargs):
"""Create profile when user is created."""
if created:
Profile.objects.create(user=instance)
@receiver(post_save, sender=User)
def save_user_profile(sender, instance, **kwargs):
"""Save profile when user is saved."""
instance.profile.save()
# apps/users/apps.py
from django.apps import AppConfig
class UsersConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'apps.users'
def ready(self):
"""Import signals when app is ready."""
import apps.users.signals
```
## Middleware
### Custom Middleware
```python
# middleware/active_user_middleware.py
import time
from django.utils.deprecation import MiddlewareMixin
class ActiveUserMiddleware(MiddlewareMixin):
"""Middleware to track active users."""
def process_request(self, request):
"""Process incoming request."""
if request.user.is_authenticated:
# Update last active time
request.user.last_active = timezone.now()
request.user.save(update_fields=['last_active'])
class RequestLoggingMiddleware(MiddlewareMixin):
"""Middleware for logging requests."""
def process_request(self, request):
"""Log request start time."""
request.start_time = time.time()
def process_response(self, request, response):
"""Log request duration."""
if hasattr(request, 'start_time'):
duration = time.time() - request.start_time
logger.info(f'{request.method} {request.path} - {response.status_code} - {duration:.3f}s')
return response
```
## Performance Optimization
### N+1 Query Prevention
```python
# Bad - N+1 queries
products = Product.objects.all()
for product in products:
print(product.category.name) # Separate query for each product
# Good - Single query with select_related
products = Product.objects.select_related('category').all()
for product in products:
print(product.category.name)
# Good - Prefetch for many-to-many
products = Product.objects.prefetch_related('tags').all()
for product in products:
for tag in product.tags.all():
print(tag.name)
```
### Database Indexing
```python
class Product(models.Model):
name = models.CharField(max_length=200, db_index=True)
slug = models.SlugField(unique=True)
category = models.ForeignKey('Category', on_delete=models.CASCADE)
created_at = models.DateTimeField(auto_now_add=True)
class Meta:
indexes = [
models.Index(fields=['name']),
models.Index(fields=['-created_at']),
models.Index(fields=['category', 'created_at']),
]
```
### Bulk Operations
```python
# Bulk create
Product.objects.bulk_create([
Product(name=f'Product {i}', price=10.00)
for i in range(1000)
])
# Bulk update
products = Product.objects.all()[:100]
for product in products:
product.is_active = True
Product.objects.bulk_update(products, ['is_active'])
# Bulk delete
Product.objects.filter(stock=0).delete()
```
## Quick Reference
| Pattern | Description |
|---------|-------------|
| Split settings | Separate dev/prod/test settings |
| Custom QuerySet | Reusable query methods |
| Service Layer | Business logic separation |
| ViewSet | REST API endpoints |
| Serializer validation | Request/response transformation |
| select_related | Foreign key optimization |
| prefetch_related | Many-to-many optimization |
| Cache first | Cache expensive operations |
| Signals | Event-driven actions |
| Middleware | Request/response processing |
Remember: Django provides many shortcuts, but for production applications, structure and organization matter more than concise code. Build for maintainability.

592
skills/django-security/SKILL.md Normal file
View File

@@ -0,0 +1,592 @@
---
name: django-security
description: Django security best practices, authentication, authorization, CSRF protection, SQL injection prevention, XSS prevention, and secure deployment configurations.
---
# Django Security Best Practices
Comprehensive security guidelines for Django applications to protect against common vulnerabilities.
## When to Activate
- Setting up Django authentication and authorization
- Implementing user permissions and roles
- Configuring production security settings
- Reviewing Django application for security issues
- Deploying Django applications to production
## Core Security Settings
### Production Settings Configuration
```python
# settings/production.py
import os
DEBUG = False # CRITICAL: Never use True in production
ALLOWED_HOSTS = os.environ.get('ALLOWED_HOSTS', '').split(',')
# Security headers
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000 # 1 year
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_HSTS_PRELOAD = True
SECURE_CONTENT_TYPE_NOSNIFF = True
SECURE_BROWSER_XSS_FILTER = True
X_FRAME_OPTIONS = 'DENY'
# HTTPS and Cookies
SESSION_COOKIE_HTTPONLY = True
CSRF_COOKIE_HTTPONLY = True
SESSION_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SAMESITE = 'Lax'
# Secret key (must be set via environment variable)
SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY')
if not SECRET_KEY:
raise ImproperlyConfigured('DJANGO_SECRET_KEY environment variable is required')
# Password validation
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
'OPTIONS': {
'min_length': 12,
}
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
```
## Authentication
### Custom User Model
```python
# apps/users/models.py
from django.contrib.auth.models import AbstractUser
from django.db import models
class User(AbstractUser):
"""Custom user model for better security."""
email = models.EmailField(unique=True)
phone = models.CharField(max_length=20, blank=True)
USERNAME_FIELD = 'email' # Use email as username
REQUIRED_FIELDS = ['username']
class Meta:
db_table = 'users'
verbose_name = 'User'
verbose_name_plural = 'Users'
def __str__(self):
return self.email
# settings/base.py
AUTH_USER_MODEL = 'users.User'
```
### Password Hashing
```python
# Django uses PBKDF2 by default. For stronger security:
PASSWORD_HASHERS = [
'django.contrib.auth.hashers.Argon2PasswordHasher',
'django.contrib.auth.hashers.PBKDF2PasswordHasher',
'django.contrib.auth.hashers.PBKDF2SHA1PasswordHasher',
'django.contrib.auth.hashers.BCryptSHA256PasswordHasher',
]
```
### Session Management
```python
# Session configuration
SESSION_ENGINE = 'django.contrib.sessions.backends.cache' # Or 'db'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 3600 * 24 * 7 # 1 week
SESSION_SAVE_EVERY_REQUEST = False
SESSION_EXPIRE_AT_BROWSER_CLOSE = False # Better UX, but less secure
```
## Authorization
### Permissions
```python
# models.py
from django.db import models
from django.contrib.auth.models import Permission
class Post(models.Model):
title = models.CharField(max_length=200)
content = models.TextField()
author = models.ForeignKey(User, on_delete=models.CASCADE)
class Meta:
permissions = [
('can_publish', 'Can publish posts'),
('can_edit_others', 'Can edit posts of others'),
]
def user_can_edit(self, user):
"""Check if user can edit this post."""
return self.author == user or user.has_perm('app.can_edit_others')
# views.py
from django.contrib.auth.mixins import LoginRequiredMixin, PermissionRequiredMixin
from django.views.generic import UpdateView
class PostUpdateView(LoginRequiredMixin, PermissionRequiredMixin, UpdateView):
model = Post
permission_required = 'app.can_edit_others'
raise_exception = True # Return 403 instead of redirect
def get_queryset(self):
"""Only allow users to edit their own posts."""
return Post.objects.filter(author=self.request.user)
```
### Custom Permissions
```python
# permissions.py
from rest_framework import permissions
class IsOwnerOrReadOnly(permissions.BasePermission):
"""Allow only owners to edit objects."""
def has_object_permission(self, request, view, obj):
# Read permissions allowed for any request
if request.method in permissions.SAFE_METHODS:
return True
# Write permissions only for owner
return obj.author == request.user
class IsAdminOrReadOnly(permissions.BasePermission):
"""Allow admins to do anything, others read-only."""
def has_permission(self, request, view):
if request.method in permissions.SAFE_METHODS:
return True
return request.user and request.user.is_staff
class IsVerifiedUser(permissions.BasePermission):
"""Allow only verified users."""
def has_permission(self, request, view):
return request.user and request.user.is_authenticated and request.user.is_verified
```
### Role-Based Access Control (RBAC)
```python
# models.py
from django.contrib.auth.models import AbstractUser, Group
class User(AbstractUser):
ROLE_CHOICES = [
('admin', 'Administrator'),
('moderator', 'Moderator'),
('user', 'Regular User'),
]
role = models.CharField(max_length=20, choices=ROLE_CHOICES, default='user')
def is_admin(self):
return self.role == 'admin' or self.is_superuser
def is_moderator(self):
return self.role in ['admin', 'moderator']
# Mixins
class AdminRequiredMixin:
"""Mixin to require admin role."""
def dispatch(self, request, *args, **kwargs):
if not request.user.is_authenticated or not request.user.is_admin():
from django.core.exceptions import PermissionDenied
raise PermissionDenied
return super().dispatch(request, *args, **kwargs)
```
## SQL Injection Prevention
### Django ORM Protection
```python
# GOOD: Django ORM automatically escapes parameters
def get_user(username):
return User.objects.get(username=username) # Safe
# GOOD: Using parameters with raw()
def search_users(query):
return User.objects.raw('SELECT * FROM users WHERE username = %s', [query])
# BAD: Never directly interpolate user input
def get_user_bad(username):
return User.objects.raw(f'SELECT * FROM users WHERE username = {username}') # VULNERABLE!
# GOOD: Using filter with proper escaping
def get_users_by_email(email):
return User.objects.filter(email__iexact=email) # Safe
# GOOD: Using Q objects for complex queries
from django.db.models import Q
def search_users_complex(query):
return User.objects.filter(
Q(username__icontains=query) |
Q(email__icontains=query)
) # Safe
```
### Extra Security with raw()
```python
# If you must use raw SQL, always use parameters
User.objects.raw(
'SELECT * FROM users WHERE email = %s AND status = %s',
[user_input_email, status]
)
```
## XSS Prevention
### Template Escaping
```django
{# Django auto-escapes variables by default - SAFE #}
{{ user_input }} {# Escaped HTML #}
{# Explicitly mark safe only for trusted content #}
{{ trusted_html|safe }} {# Not escaped #}
{# Use template filters for safe HTML #}
{{ user_input|escape }} {# Same as default #}
{{ user_input|striptags }} {# Remove all HTML tags #}
{# JavaScript escaping #}
<script>
var username = {{ username|escapejs }};
</script>
```
### Safe String Handling
```python
from django.utils.safestring import mark_safe
from django.utils.html import escape
# BAD: Never mark user input as safe without escaping
def render_bad(user_input):
return mark_safe(user_input) # VULNERABLE!
# GOOD: Escape first, then mark safe
def render_good(user_input):
return mark_safe(escape(user_input))
# GOOD: Use format_html for HTML with variables
from django.utils.html import format_html
def greet_user(username):
return format_html('<span class="user">{}</span>', escape(username))
```
### HTTP Headers
```python
# settings.py
SECURE_CONTENT_TYPE_NOSNIFF = True # Prevent MIME sniffing
SECURE_BROWSER_XSS_FILTER = True # Enable XSS filter
X_FRAME_OPTIONS = 'DENY' # Prevent clickjacking
# Custom middleware
from django.conf import settings
class SecurityHeaderMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
response['X-Content-Type-Options'] = 'nosniff'
response['X-Frame-Options'] = 'DENY'
response['X-XSS-Protection'] = '1; mode=block'
response['Content-Security-Policy'] = "default-src 'self'"
return response
```
## CSRF Protection
### Default CSRF Protection
```python
# settings.py - CSRF is enabled by default
CSRF_COOKIE_SECURE = True # Only send over HTTPS
CSRF_COOKIE_HTTPONLY = True # Prevent JavaScript access
CSRF_COOKIE_SAMESITE = 'Lax' # Prevent CSRF in some cases
CSRF_TRUSTED_ORIGINS = ['https://example.com'] # Trusted domains
# Template usage
<form method="post">
{% csrf_token %}
{{ form.as_p }}
<button type="submit">Submit</button>
</form>
# AJAX requests
function getCookie(name) {
let cookieValue = null;
if (document.cookie && document.cookie !== '') {
const cookies = document.cookie.split(';');
for (let i = 0; i < cookies.length; i++) {
const cookie = cookies[i].trim();
if (cookie.substring(0, name.length + 1) === (name + '=')) {
cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
break;
}
}
}
return cookieValue;
}
fetch('/api/endpoint/', {
method: 'POST',
headers: {
'X-CSRFToken': getCookie('csrftoken'),
'Content-Type': 'application/json',
},
body: JSON.stringify(data)
});
```
### Exempting Views (Use Carefully)
```python
from django.views.decorators.csrf import csrf_exempt
@csrf_exempt # Only use when absolutely necessary!
def webhook_view(request):
# Webhook from external service
pass
```
## File Upload Security
### File Validation
```python
import os
from django.core.exceptions import ValidationError
def validate_file_extension(value):
"""Validate file extension."""
ext = os.path.splitext(value.name)[1]
valid_extensions = ['.jpg', '.jpeg', '.png', '.gif', '.pdf']
if not ext.lower() in valid_extensions:
raise ValidationError('Unsupported file extension.')
def validate_file_size(value):
"""Validate file size (max 5MB)."""
filesize = value.size
if filesize > 5 * 1024 * 1024:
raise ValidationError('File too large. Max size is 5MB.')
# models.py
class Document(models.Model):
file = models.FileField(
upload_to='documents/',
validators=[validate_file_extension, validate_file_size]
)
```
### Secure File Storage
```python
# settings.py
MEDIA_ROOT = '/var/www/media/'
MEDIA_URL = '/media/'
# Use a separate domain for media in production
MEDIA_DOMAIN = 'https://media.example.com'
# Don't serve user uploads directly
# Use whitenoise or a CDN for static files
# Use a separate server or S3 for media files
```
## API Security
### Rate Limiting
```python
# settings.py
REST_FRAMEWORK = {
'DEFAULT_THROTTLE_CLASSES': [
'rest_framework.throttling.AnonRateThrottle',
'rest_framework.throttling.UserRateThrottle'
],
'DEFAULT_THROTTLE_RATES': {
'anon': '100/day',
'user': '1000/day',
'upload': '10/hour',
}
}
# Custom throttle
from rest_framework.throttling import UserRateThrottle
class BurstRateThrottle(UserRateThrottle):
scope = 'burst'
rate = '60/min'
class SustainedRateThrottle(UserRateThrottle):
scope = 'sustained'
rate = '1000/day'
```
### Authentication for APIs
```python
# settings.py
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.SessionAuthentication',
'rest_framework_simplejwt.authentication.JWTAuthentication',
],
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.IsAuthenticated',
],
}
# views.py
from rest_framework.decorators import api_view, permission_classes
from rest_framework.permissions import IsAuthenticated
@api_view(['GET', 'POST'])
@permission_classes([IsAuthenticated])
def protected_view(request):
return Response({'message': 'You are authenticated'})
```
## Security Headers
### Content Security Policy
```python
# settings.py
CSP_DEFAULT_SRC = "'self'"
CSP_SCRIPT_SRC = "'self' https://cdn.example.com"
CSP_STYLE_SRC = "'self' 'unsafe-inline'"
CSP_IMG_SRC = "'self' data: https:"
CSP_CONNECT_SRC = "'self' https://api.example.com"
# Middleware
class CSPMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
response['Content-Security-Policy'] = (
f"default-src {CSP_DEFAULT_SRC}; "
f"script-src {CSP_SCRIPT_SRC}; "
f"style-src {CSP_STYLE_SRC}; "
f"img-src {CSP_IMG_SRC}; "
f"connect-src {CSP_CONNECT_SRC}"
)
return response
```
## Environment Variables
### Managing Secrets
```python
# Use python-decouple or django-environ
import environ
env = environ.Env(
# set casting, default value
DEBUG=(bool, False)
)
# reading .env file
environ.Env.read_env()
SECRET_KEY = env('DJANGO_SECRET_KEY')
DATABASE_URL = env('DATABASE_URL')
ALLOWED_HOSTS = env.list('ALLOWED_HOSTS')
# .env file (never commit this)
DEBUG=False
SECRET_KEY=your-secret-key-here
DATABASE_URL=postgresql://user:password@localhost:5432/dbname
ALLOWED_HOSTS=example.com,www.example.com
```
## Logging Security Events
```python
# settings.py
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'file': {
'level': 'WARNING',
'class': 'logging.FileHandler',
'filename': '/var/log/django/security.log',
},
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler',
},
},
'loggers': {
'django.security': {
'handlers': ['file', 'console'],
'level': 'WARNING',
'propagate': True,
},
'django.request': {
'handlers': ['file'],
'level': 'ERROR',
'propagate': False,
},
},
}
```
## Quick Security Checklist
| Check | Description |
|-------|-------------|
| `DEBUG = False` | Never run with DEBUG in production |
| HTTPS only | Force SSL, secure cookies |
| Strong secrets | Use environment variables for SECRET_KEY |
| Password validation | Enable all password validators |
| CSRF protection | Enabled by default, don't disable |
| XSS prevention | Django auto-escapes, don't use `&#124;safe` with user input |
| SQL injection | Use ORM, never concatenate strings in queries |
| File uploads | Validate file type and size |
| Rate limiting | Throttle API endpoints |
| Security headers | CSP, X-Frame-Options, HSTS |
| Logging | Log security events |
| Updates | Keep Django and dependencies updated |
Remember: Security is a process, not a product. Regularly review and update your security practices.

728
skills/django-tdd/SKILL.md Normal file
View File

@@ -0,0 +1,728 @@
---
name: django-tdd
description: Django testing strategies with pytest-django, TDD methodology, factory_boy, mocking, coverage, and testing Django REST Framework APIs.
---
# Django Testing with TDD
Test-driven development for Django applications using pytest, factory_boy, and Django REST Framework.
## When to Activate
- Writing new Django applications
- Implementing Django REST Framework APIs
- Testing Django models, views, and serializers
- Setting up testing infrastructure for Django projects
## TDD Workflow for Django
### Red-Green-Refactor Cycle
```python
# Step 1: RED - Write failing test
def test_user_creation():
user = User.objects.create_user(email='test@example.com', password='testpass123')
assert user.email == 'test@example.com'
assert user.check_password('testpass123')
assert not user.is_staff
# Step 2: GREEN - Make test pass
# Create User model or factory
# Step 3: REFACTOR - Improve while keeping tests green
```
## Setup
### pytest Configuration
```ini
# pytest.ini
[pytest]
DJANGO_SETTINGS_MODULE = config.settings.test
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
--reuse-db
--nomigrations
--cov=apps
--cov-report=html
--cov-report=term-missing
--strict-markers
markers =
slow: marks tests as slow
integration: marks tests as integration tests
```
### Test Settings
```python
# config/settings/test.py
from .base import *
DEBUG = True
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': ':memory:',
}
}
# Disable migrations for speed
class DisableMigrations:
def __contains__(self, item):
return True
def __getitem__(self, item):
return None
MIGRATION_MODULES = DisableMigrations()
# Faster password hashing
PASSWORD_HASHERS = [
'django.contrib.auth.hashers.MD5PasswordHasher',
]
# Email backend
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
# Celery always eager
CELERY_TASK_ALWAYS_EAGER = True
CELERY_TASK_EAGER_PROPAGATES = True
```
### conftest.py
```python
# tests/conftest.py
import pytest
from django.utils import timezone
from django.contrib.auth import get_user_model
User = get_user_model()
@pytest.fixture(autouse=True)
def timezone_settings(settings):
"""Ensure consistent timezone."""
settings.TIME_ZONE = 'UTC'
@pytest.fixture
def user(db):
"""Create a test user."""
return User.objects.create_user(
email='test@example.com',
password='testpass123',
username='testuser'
)
@pytest.fixture
def admin_user(db):
"""Create an admin user."""
return User.objects.create_superuser(
email='admin@example.com',
password='adminpass123',
username='admin'
)
@pytest.fixture
def authenticated_client(client, user):
"""Return authenticated client."""
client.force_login(user)
return client
@pytest.fixture
def api_client():
"""Return DRF API client."""
from rest_framework.test import APIClient
return APIClient()
@pytest.fixture
def authenticated_api_client(api_client, user):
"""Return authenticated API client."""
api_client.force_authenticate(user=user)
return api_client
```
## Factory Boy
### Factory Setup
```python
# tests/factories.py
import factory
from factory import fuzzy
from datetime import datetime, timedelta
from django.contrib.auth import get_user_model
from apps.products.models import Product, Category
User = get_user_model()
class UserFactory(factory.django.DjangoModelFactory):
"""Factory for User model."""
class Meta:
model = User
email = factory.Sequence(lambda n: f"user{n}@example.com")
username = factory.Sequence(lambda n: f"user{n}")
password = factory.PostGenerationMethodCall('set_password', 'testpass123')
first_name = factory.Faker('first_name')
last_name = factory.Faker('last_name')
is_active = True
class CategoryFactory(factory.django.DjangoModelFactory):
"""Factory for Category model."""
class Meta:
model = Category
name = factory.Faker('word')
slug = factory.LazyAttribute(lambda obj: obj.name.lower())
description = factory.Faker('text')
class ProductFactory(factory.django.DjangoModelFactory):
"""Factory for Product model."""
class Meta:
model = Product
name = factory.Faker('sentence', nb_words=3)
slug = factory.LazyAttribute(lambda obj: obj.name.lower().replace(' ', '-'))
description = factory.Faker('text')
price = fuzzy.FuzzyDecimal(10.00, 1000.00, 2)
stock = fuzzy.FuzzyInteger(0, 100)
is_active = True
category = factory.SubFactory(CategoryFactory)
created_by = factory.SubFactory(UserFactory)
@factory.post_generation
def tags(self, create, extracted, **kwargs):
"""Add tags to product."""
if not create:
return
if extracted:
for tag in extracted:
self.tags.add(tag)
```
### Using Factories
```python
# tests/test_models.py
import pytest
from tests.factories import ProductFactory, UserFactory
def test_product_creation():
"""Test product creation using factory."""
product = ProductFactory(price=100.00, stock=50)
assert product.price == 100.00
assert product.stock == 50
assert product.is_active is True
def test_product_with_tags():
"""Test product with tags."""
tags = [TagFactory(name='electronics'), TagFactory(name='new')]
product = ProductFactory(tags=tags)
assert product.tags.count() == 2
def test_multiple_products():
"""Test creating multiple products."""
products = ProductFactory.create_batch(10)
assert len(products) == 10
```
## Model Testing
### Model Tests
```python
# tests/test_models.py
import pytest
from django.core.exceptions import ValidationError
from tests.factories import UserFactory, ProductFactory
class TestUserModel:
"""Test User model."""
def test_create_user(self, db):
"""Test creating a regular user."""
user = UserFactory(email='test@example.com')
assert user.email == 'test@example.com'
assert user.check_password('testpass123')
assert not user.is_staff
assert not user.is_superuser
def test_create_superuser(self, db):
"""Test creating a superuser."""
user = UserFactory(
email='admin@example.com',
is_staff=True,
is_superuser=True
)
assert user.is_staff
assert user.is_superuser
def test_user_str(self, db):
"""Test user string representation."""
user = UserFactory(email='test@example.com')
assert str(user) == 'test@example.com'
class TestProductModel:
"""Test Product model."""
def test_product_creation(self, db):
"""Test creating a product."""
product = ProductFactory()
assert product.id is not None
assert product.is_active is True
assert product.created_at is not None
def test_product_slug_generation(self, db):
"""Test automatic slug generation."""
product = ProductFactory(name='Test Product')
assert product.slug == 'test-product'
def test_product_price_validation(self, db):
"""Test price cannot be negative."""
product = ProductFactory(price=-10)
with pytest.raises(ValidationError):
product.full_clean()
def test_product_manager_active(self, db):
"""Test active manager method."""
ProductFactory.create_batch(5, is_active=True)
ProductFactory.create_batch(3, is_active=False)
active_count = Product.objects.active().count()
assert active_count == 5
def test_product_stock_management(self, db):
"""Test stock management."""
product = ProductFactory(stock=10)
product.reduce_stock(5)
product.refresh_from_db()
assert product.stock == 5
with pytest.raises(ValueError):
product.reduce_stock(10) # Not enough stock
```
## View Testing
### Django View Testing
```python
# tests/test_views.py
import pytest
from django.urls import reverse
from tests.factories import ProductFactory, UserFactory
class TestProductViews:
"""Test product views."""
def test_product_list(self, client, db):
"""Test product list view."""
ProductFactory.create_batch(10)
response = client.get(reverse('products:list'))
assert response.status_code == 200
assert len(response.context['products']) == 10
def test_product_detail(self, client, db):
"""Test product detail view."""
product = ProductFactory()
response = client.get(reverse('products:detail', kwargs={'slug': product.slug}))
assert response.status_code == 200
assert response.context['product'] == product
def test_product_create_requires_login(self, client, db):
"""Test product creation requires authentication."""
response = client.get(reverse('products:create'))
assert response.status_code == 302
assert response.url.startswith('/accounts/login/')
def test_product_create_authenticated(self, authenticated_client, db):
"""Test product creation as authenticated user."""
response = authenticated_client.get(reverse('products:create'))
assert response.status_code == 200
def test_product_create_post(self, authenticated_client, db, category):
"""Test creating a product via POST."""
data = {
'name': 'Test Product',
'description': 'A test product',
'price': '99.99',
'stock': 10,
'category': category.id,
}
response = authenticated_client.post(reverse('products:create'), data)
assert response.status_code == 302
assert Product.objects.filter(name='Test Product').exists()
```
## DRF API Testing
### Serializer Testing
```python
# tests/test_serializers.py
import pytest
from rest_framework.exceptions import ValidationError
from apps.products.serializers import ProductSerializer
from tests.factories import ProductFactory
class TestProductSerializer:
"""Test ProductSerializer."""
def test_serialize_product(self, db):
"""Test serializing a product."""
product = ProductFactory()
serializer = ProductSerializer(product)
data = serializer.data
assert data['id'] == product.id
assert data['name'] == product.name
assert data['price'] == str(product.price)
def test_deserialize_product(self, db):
"""Test deserializing product data."""
data = {
'name': 'Test Product',
'description': 'Test description',
'price': '99.99',
'stock': 10,
'category': 1,
}
serializer = ProductSerializer(data=data)
assert serializer.is_valid()
product = serializer.save()
assert product.name == 'Test Product'
assert float(product.price) == 99.99
def test_price_validation(self, db):
"""Test price validation."""
data = {
'name': 'Test Product',
'price': '-10.00',
'stock': 10,
}
serializer = ProductSerializer(data=data)
assert not serializer.is_valid()
assert 'price' in serializer.errors
def test_stock_validation(self, db):
"""Test stock cannot be negative."""
data = {
'name': 'Test Product',
'price': '99.99',
'stock': -5,
}
serializer = ProductSerializer(data=data)
assert not serializer.is_valid()
assert 'stock' in serializer.errors
```
### API ViewSet Testing
```python
# tests/test_api.py
import pytest
from rest_framework.test import APIClient
from rest_framework import status
from django.urls import reverse
from tests.factories import ProductFactory, UserFactory
class TestProductAPI:
"""Test Product API endpoints."""
@pytest.fixture
def api_client(self):
"""Return API client."""
return APIClient()
def test_list_products(self, api_client, db):
"""Test listing products."""
ProductFactory.create_batch(10)
url = reverse('api:product-list')
response = api_client.get(url)
assert response.status_code == status.HTTP_200_OK
assert response.data['count'] == 10
def test_retrieve_product(self, api_client, db):
"""Test retrieving a product."""
product = ProductFactory()
url = reverse('api:product-detail', kwargs={'pk': product.id})
response = api_client.get(url)
assert response.status_code == status.HTTP_200_OK
assert response.data['id'] == product.id
def test_create_product_unauthorized(self, api_client, db):
"""Test creating product without authentication."""
url = reverse('api:product-list')
data = {'name': 'Test Product', 'price': '99.99'}
response = api_client.post(url, data)
assert response.status_code == status.HTTP_401_UNAUTHORIZED
def test_create_product_authorized(self, authenticated_api_client, db):
"""Test creating product as authenticated user."""
url = reverse('api:product-list')
data = {
'name': 'Test Product',
'description': 'Test',
'price': '99.99',
'stock': 10,
}
response = authenticated_api_client.post(url, data)
assert response.status_code == status.HTTP_201_CREATED
assert response.data['name'] == 'Test Product'
def test_update_product(self, authenticated_api_client, db):
"""Test updating a product."""
product = ProductFactory(created_by=authenticated_api_client.user)
url = reverse('api:product-detail', kwargs={'pk': product.id})
data = {'name': 'Updated Product'}
response = authenticated_api_client.patch(url, data)
assert response.status_code == status.HTTP_200_OK
assert response.data['name'] == 'Updated Product'
def test_delete_product(self, authenticated_api_client, db):
"""Test deleting a product."""
product = ProductFactory(created_by=authenticated_api_client.user)
url = reverse('api:product-detail', kwargs={'pk': product.id})
response = authenticated_api_client.delete(url)
assert response.status_code == status.HTTP_204_NO_CONTENT
def test_filter_products_by_price(self, api_client, db):
"""Test filtering products by price."""
ProductFactory(price=50)
ProductFactory(price=150)
url = reverse('api:product-list')
response = api_client.get(url, {'price_min': 100})
assert response.status_code == status.HTTP_200_OK
assert response.data['count'] == 1
def test_search_products(self, api_client, db):
"""Test searching products."""
ProductFactory(name='Apple iPhone')
ProductFactory(name='Samsung Galaxy')
url = reverse('api:product-list')
response = api_client.get(url, {'search': 'Apple'})
assert response.status_code == status.HTTP_200_OK
assert response.data['count'] == 1
```
## Mocking and Patching
### Mocking External Services
```python
# tests/test_views.py
from unittest.mock import patch, Mock
import pytest
class TestPaymentView:
"""Test payment view with mocked payment gateway."""
@patch('apps.payments.services.stripe')
def test_successful_payment(self, mock_stripe, client, user, product):
"""Test successful payment with mocked Stripe."""
# Configure mock
mock_stripe.Charge.create.return_value = {
'id': 'ch_123',
'status': 'succeeded',
'amount': 9999,
}
client.force_login(user)
response = client.post(reverse('payments:process'), {
'product_id': product.id,
'token': 'tok_visa',
})
assert response.status_code == 302
mock_stripe.Charge.create.assert_called_once()
@patch('apps.payments.services.stripe')
def test_failed_payment(self, mock_stripe, client, user, product):
"""Test failed payment."""
mock_stripe.Charge.create.side_effect = Exception('Card declined')
client.force_login(user)
response = client.post(reverse('payments:process'), {
'product_id': product.id,
'token': 'tok_visa',
})
assert response.status_code == 302
assert 'error' in response.url
```
### Mocking Email Sending
```python
# tests/test_email.py
from django.core import mail
from django.test import override_settings
@override_settings(EMAIL_BACKEND='django.core.mail.backends.locmem.EmailBackend')
def test_order_confirmation_email(db, order):
"""Test order confirmation email."""
order.send_confirmation_email()
assert len(mail.outbox) == 1
assert order.user.email in mail.outbox[0].to
assert 'Order Confirmation' in mail.outbox[0].subject
```
## Integration Testing
### Full Flow Testing
```python
# tests/test_integration.py
import pytest
from django.urls import reverse
from tests.factories import UserFactory, ProductFactory
class TestCheckoutFlow:
"""Test complete checkout flow."""
def test_guest_to_purchase_flow(self, client, db):
"""Test complete flow from guest to purchase."""
# Step 1: Register
response = client.post(reverse('users:register'), {
'email': 'test@example.com',
'password': 'testpass123',
'password_confirm': 'testpass123',
})
assert response.status_code == 302
# Step 2: Login
response = client.post(reverse('users:login'), {
'email': 'test@example.com',
'password': 'testpass123',
})
assert response.status_code == 302
# Step 3: Browse products
product = ProductFactory(price=100)
response = client.get(reverse('products:detail', kwargs={'slug': product.slug}))
assert response.status_code == 200
# Step 4: Add to cart
response = client.post(reverse('cart:add'), {
'product_id': product.id,
'quantity': 1,
})
assert response.status_code == 302
# Step 5: Checkout
response = client.get(reverse('checkout:review'))
assert response.status_code == 200
assert product.name in response.content.decode()
# Step 6: Complete purchase
with patch('apps.checkout.services.process_payment') as mock_payment:
mock_payment.return_value = True
response = client.post(reverse('checkout:complete'))
assert response.status_code == 302
assert Order.objects.filter(user__email='test@example.com').exists()
```
## Testing Best Practices
### DO
- **Use factories**: Instead of manual object creation
- **One assertion per test**: Keep tests focused
- **Descriptive test names**: `test_user_cannot_delete_others_post`
- **Test edge cases**: Empty inputs, None values, boundary conditions
- **Mock external services**: Don't depend on external APIs
- **Use fixtures**: Eliminate duplication
- **Test permissions**: Ensure authorization works
- **Keep tests fast**: Use `--reuse-db` and `--nomigrations`
### DON'T
- **Don't test Django internals**: Trust Django to work
- **Don't test third-party code**: Trust libraries to work
- **Don't ignore failing tests**: All tests must pass
- **Don't make tests dependent**: Tests should run in any order
- **Don't over-mock**: Mock only external dependencies
- **Don't test private methods**: Test public interface
- **Don't use production database**: Always use test database
## Coverage
### Coverage Configuration
```bash
# Run tests with coverage
pytest --cov=apps --cov-report=html --cov-report=term-missing
# Generate HTML report
open htmlcov/index.html
```
### Coverage Goals
| Component | Target Coverage |
|-----------|-----------------|
| Models | 90%+ |
| Serializers | 85%+ |
| Views | 80%+ |
| Services | 90%+ |
| Utilities | 80%+ |
| Overall | 80%+ |
## Quick Reference
| Pattern | Usage |
|---------|-------|
| `@pytest.mark.django_db` | Enable database access |
| `client` | Django test client |
| `api_client` | DRF API client |
| `factory.create_batch(n)` | Create multiple objects |
| `patch('module.function')` | Mock external dependencies |
| `override_settings` | Temporarily change settings |
| `force_authenticate()` | Bypass authentication in tests |
| `assertRedirects` | Check for redirects |
| `assertTemplateUsed` | Verify template usage |
| `mail.outbox` | Check sent emails |
Remember: Tests are documentation. Good tests explain how your code should work. Keep them simple, readable, and maintainable.

468
skills/django-verification/SKILL.md Normal file
View File

@@ -0,0 +1,468 @@
---
name: django-verification
description: "Verification loop for Django projects: migrations, linting, tests with coverage, security scans, and deployment readiness checks before release or PR."
---
# Django Verification Loop
Run before PRs, after major changes, and pre-deploy to ensure Django application quality and security.
## When to Activate
- Before opening a pull request for a Django project
- After major model changes, migration updates, or dependency upgrades
- Pre-deployment verification for staging or production
- Running full environment → lint → test → security → deploy readiness pipeline
- Validating migration safety and test coverage
## Phase 1: Environment Check
```bash
# Verify Python version
python --version # Should match project requirements
# Check virtual environment
which python
pip list --outdated
# Verify environment variables
python -c "import os; import environ; print('DJANGO_SECRET_KEY set' if os.environ.get('DJANGO_SECRET_KEY') else 'MISSING: DJANGO_SECRET_KEY')"
```
If environment is misconfigured, stop and fix.
## Phase 2: Code Quality & Formatting
```bash
# Type checking
mypy . --config-file pyproject.toml
# Linting with ruff
ruff check . --fix
# Formatting with black
black . --check
black . # Auto-fix
# Import sorting
isort . --check-only
isort . # Auto-fix
# Django-specific checks
python manage.py check --deploy
```
Common issues:
- Missing type hints on public functions
- PEP 8 formatting violations
- Unsorted imports
- Debug settings left in production configuration
## Phase 3: Migrations
```bash
# Check for unapplied migrations
python manage.py showmigrations
# Create missing migrations
python manage.py makemigrations --check
# Dry-run migration application
python manage.py migrate --plan
# Apply migrations (test environment)
python manage.py migrate
# Check for migration conflicts
python manage.py makemigrations --merge # Only if conflicts exist
```
Report:
- Number of pending migrations
- Any migration conflicts
- Model changes without migrations
## Phase 4: Tests + Coverage
```bash
# Run all tests with pytest
pytest --cov=apps --cov-report=html --cov-report=term-missing --reuse-db
# Run specific app tests
pytest apps/users/tests/
# Run with markers
pytest -m "not slow" # Skip slow tests
pytest -m integration # Only integration tests
# Coverage report
open htmlcov/index.html
```
Report:
- Total tests: X passed, Y failed, Z skipped
- Overall coverage: XX%
- Per-app coverage breakdown
Coverage targets:
| Component | Target |
|-----------|--------|
| Models | 90%+ |
| Serializers | 85%+ |
| Views | 80%+ |
| Services | 90%+ |
| Overall | 80%+ |
## Phase 5: Security Scan
```bash
# Dependency vulnerabilities
pip-audit
safety check --full-report
# Django security checks
python manage.py check --deploy
# Bandit security linter
bandit -r . -f json -o bandit-report.json
# Secret scanning (if gitleaks is installed)
gitleaks detect --source . --verbose
# Environment variable check
python -c "from django.core.exceptions import ImproperlyConfigured; from django.conf import settings; settings.DEBUG"
```
Report:
- Vulnerable dependencies found
- Security configuration issues
- Hardcoded secrets detected
- DEBUG mode status (should be False in production)
## Phase 6: Django Management Commands
```bash
# Check for model issues
python manage.py check
# Collect static files
python manage.py collectstatic --noinput --clear
# Create superuser (if needed for tests)
echo "from apps.users.models import User; User.objects.create_superuser('admin@example.com', 'admin')" | python manage.py shell
# Database integrity
python manage.py check --database default
# Cache verification (if using Redis)
python -c "from django.core.cache import cache; cache.set('test', 'value', 10); print(cache.get('test'))"
```
## Phase 7: Performance Checks
```bash
# Django Debug Toolbar output (check for N+1 queries)
# Run in dev mode with DEBUG=True and access a page
# Look for duplicate queries in SQL panel
# Query count analysis
django-admin debugsqlshell # If django-debug-sqlshell installed
# Check for missing indexes
python manage.py shell << EOF
from django.db import connection
with connection.cursor() as cursor:
cursor.execute("SELECT table_name, index_name FROM information_schema.statistics WHERE table_schema = 'public'")
print(cursor.fetchall())
EOF
```
Report:
- Number of queries per page (should be < 50 for typical pages)
- Missing database indexes
- Duplicate queries detected
## Phase 8: Static Assets
```bash
# Check for npm dependencies (if using npm)
npm audit
npm audit fix
# Build static files (if using webpack/vite)
npm run build
# Verify static files
ls -la staticfiles/
python manage.py findstatic css/style.css
```
## Phase 9: Configuration Review
```python
# Run in Python shell to verify settings
python manage.py shell << EOF
from django.conf import settings
import os
# Critical checks
checks = {
'DEBUG is False': not settings.DEBUG,
'SECRET_KEY set': bool(settings.SECRET_KEY and len(settings.SECRET_KEY) > 30),
'ALLOWED_HOSTS set': len(settings.ALLOWED_HOSTS) > 0,
'HTTPS enabled': getattr(settings, 'SECURE_SSL_REDIRECT', False),
'HSTS enabled': getattr(settings, 'SECURE_HSTS_SECONDS', 0) > 0,
'Database configured': settings.DATABASES['default']['ENGINE'] != 'django.db.backends.sqlite3',
}
for check, result in checks.items():
status = '✓' if result else '✗'
print(f"{status} {check}")
EOF
```
## Phase 10: Logging Configuration
```bash
# Test logging output
python manage.py shell << EOF
import logging
logger = logging.getLogger('django')
logger.warning('Test warning message')
logger.error('Test error message')
EOF
# Check log files (if configured)
tail -f /var/log/django/django.log
```
## Phase 11: API Documentation (if DRF)
```bash
# Generate schema
python manage.py generateschema --format openapi-json > schema.json
# Validate schema
# Check if schema.json is valid JSON
python -c "import json; json.load(open('schema.json'))"
# Access Swagger UI (if using drf-yasg)
# Visit http://localhost:8000/swagger/ in browser
```
## Phase 12: Diff Review
```bash
# Show diff statistics
git diff --stat
# Show actual changes
git diff
# Show changed files
git diff --name-only
# Check for common issues
git diff | grep -i "todo\|fixme\|hack\|xxx"
git diff | grep "print(" # Debug statements
git diff | grep "DEBUG = True" # Debug mode
git diff | grep "import pdb" # Debugger
```
Checklist:
- No debugging statements (print, pdb, breakpoint())
- No TODO/FIXME comments in critical code
- No hardcoded secrets or credentials
- Database migrations included for model changes
- Configuration changes documented
- Error handling present for external calls
- Transaction management where needed
## Output Template
```
DJANGO VERIFICATION REPORT
==========================
Phase 1: Environment Check
✓ Python 3.11.5
✓ Virtual environment active
✓ All environment variables set
Phase 2: Code Quality
✓ mypy: No type errors
✗ ruff: 3 issues found (auto-fixed)
✓ black: No formatting issues
✓ isort: Imports properly sorted
✓ manage.py check: No issues
Phase 3: Migrations
✓ No unapplied migrations
✓ No migration conflicts
✓ All models have migrations
Phase 4: Tests + Coverage
Tests: 247 passed, 0 failed, 5 skipped
Coverage:
Overall: 87%
users: 92%
products: 89%
orders: 85%
payments: 91%
Phase 5: Security Scan
✗ pip-audit: 2 vulnerabilities found (fix required)
✓ safety check: No issues
✓ bandit: No security issues
✓ No secrets detected
✓ DEBUG = False
Phase 6: Django Commands
✓ collectstatic completed
✓ Database integrity OK
✓ Cache backend reachable
Phase 7: Performance
✓ No N+1 queries detected
✓ Database indexes configured
✓ Query count acceptable
Phase 8: Static Assets
✓ npm audit: No vulnerabilities
✓ Assets built successfully
✓ Static files collected
Phase 9: Configuration
✓ DEBUG = False
✓ SECRET_KEY configured
✓ ALLOWED_HOSTS set
✓ HTTPS enabled
✓ HSTS enabled
✓ Database configured
Phase 10: Logging
✓ Logging configured
✓ Log files writable
Phase 11: API Documentation
✓ Schema generated
✓ Swagger UI accessible
Phase 12: Diff Review
Files changed: 12
+450, -120 lines
✓ No debug statements
✓ No hardcoded secrets
✓ Migrations included
RECOMMENDATION: ⚠️ Fix pip-audit vulnerabilities before deploying
NEXT STEPS:
1. Update vulnerable dependencies
2. Re-run security scan
3. Deploy to staging for final testing
```
## Pre-Deployment Checklist
- [ ] All tests passing
- [ ] Coverage ≥ 80%
- [ ] No security vulnerabilities
- [ ] No unapplied migrations
- [ ] DEBUG = False in production settings
- [ ] SECRET_KEY properly configured
- [ ] ALLOWED_HOSTS set correctly
- [ ] Database backups enabled
- [ ] Static files collected and served
- [ ] Logging configured and working
- [ ] Error monitoring (Sentry, etc.) configured
- [ ] CDN configured (if applicable)
- [ ] Redis/cache backend configured
- [ ] Celery workers running (if applicable)
- [ ] HTTPS/SSL configured
- [ ] Environment variables documented
## Continuous Integration
### GitHub Actions Example
```yaml
# .github/workflows/django-verification.yml
name: Django Verification
on: [push, pull_request]
jobs:
verify:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:14
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install ruff black mypy pytest pytest-django pytest-cov bandit safety pip-audit
- name: Code quality checks
run: |
ruff check .
black . --check
isort . --check-only
mypy .
- name: Security scan
run: |
bandit -r . -f json -o bandit-report.json
safety check --full-report
pip-audit
- name: Run tests
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/test
DJANGO_SECRET_KEY: test-secret-key
run: |
pytest --cov=apps --cov-report=xml --cov-report=term-missing
- name: Upload coverage
uses: codecov/codecov-action@v3
```
## Quick Reference
| Check | Command |
|-------|---------|
| Environment | `python --version` |
| Type checking | `mypy .` |
| Linting | `ruff check .` |
| Formatting | `black . --check` |
| Migrations | `python manage.py makemigrations --check` |
| Tests | `pytest --cov=apps` |
| Security | `pip-audit && bandit -r .` |
| Django check | `python manage.py check --deploy` |
| Collectstatic | `python manage.py collectstatic --noinput` |
| Diff stats | `git diff --stat` |
Remember: Automated verification catches common issues but doesn't replace manual code review and testing in staging environment.

51
skills/docker-devops.md Normal file
View File

@@ -0,0 +1,51 @@
# Docker & DevOps Skill
## System Configuration
- Docker 29.2.1 installed
- Docker Compose plugin available
- User 'ren' added to docker group
- AMD GPU (RX 6800 XT) with ROCm support for GPU containers
## Common Commands
### Docker Basics
```bash
docker ps # List running containers
docker ps -a # List all containers
docker images # List images
docker-compose up # Start services
docker-compose up -d # Start in background
docker-compose down # Stop services
docker-compose build # Build images
docker logs <container> # View logs
docker exec -it <container> bash # Enter container
```
### ROCm GPU Support
```bash
# Run with AMD GPU support
docker run --device=/dev/kfd --device=/dev/dri --group-add video -it rocm/rocm-terminal
# Docker Compose with GPU
services:
app:
devices:
- /dev/kfd
- /dev/dri
group_add:
- video
environment:
- HSA_OVERRIDE_GFX_VERSION=10.3.0
```
## Best Practices
- Use multi-stage builds for smaller images
- Always specify versions in FROM statements
- Use .dockerignore to exclude unnecessary files
- Mount volumes for persistent data
- Use docker-compose for multi-container apps
## Common Issues
- If permission denied: user is in docker group but may need re-login
- GPU not visible: check ROCm installation with `rocminfo`
- Port conflicts: use `docker ps` to check used ports

363
skills/docker-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,363 @@
---
name: docker-patterns
description: Docker and Docker Compose patterns for local development, container security, networking, volume strategies, and multi-service orchestration.
---
# Docker Patterns
Docker and Docker Compose best practices for containerized development.
## When to Activate
- Setting up Docker Compose for local development
- Designing multi-container architectures
- Troubleshooting container networking or volume issues
- Reviewing Dockerfiles for security and size
- Migrating from local dev to containerized workflow
## Docker Compose for Local Development
### Standard Web App Stack
```yaml
# docker-compose.yml
services:
app:
build:
context: .
target: dev # Use dev stage of multi-stage Dockerfile
ports:
- "3000:3000"
volumes:
- .:/app # Bind mount for hot reload
- /app/node_modules # Anonymous volume -- preserves container deps
environment:
- DATABASE_URL=postgres://postgres:postgres@db:5432/app_dev
- REDIS_URL=redis://redis:6379/0
- NODE_ENV=development
depends_on:
db:
condition: service_healthy
redis:
condition: service_started
command: npm run dev
db:
image: postgres:16-alpine
ports:
- "5432:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app_dev
volumes:
- pgdata:/var/lib/postgresql/data
- ./scripts/init-db.sql:/docker-entrypoint-initdb.d/init.sql
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 3s
retries: 5
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redisdata:/data
mailpit: # Local email testing
image: axllent/mailpit
ports:
- "8025:8025" # Web UI
- "1025:1025" # SMTP
volumes:
pgdata:
redisdata:
```
### Development vs Production Dockerfile
```dockerfile
# Stage: dependencies
FROM node:22-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
# Stage: dev (hot reload, debug tools)
FROM node:22-alpine AS dev
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["npm", "run", "dev"]
# Stage: build
FROM node:22-alpine AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build && npm prune --production
# Stage: production (minimal image)
FROM node:22-alpine AS production
WORKDIR /app
RUN addgroup -g 1001 -S appgroup && adduser -S appuser -u 1001
USER appuser
COPY --from=build --chown=appuser:appgroup /app/dist ./dist
COPY --from=build --chown=appuser:appgroup /app/node_modules ./node_modules
COPY --from=build --chown=appuser:appgroup /app/package.json ./
ENV NODE_ENV=production
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s CMD wget -qO- http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]
```
### Override Files
```yaml
# docker-compose.override.yml (auto-loaded, dev-only settings)
services:
app:
environment:
- DEBUG=app:*
- LOG_LEVEL=debug
ports:
- "9229:9229" # Node.js debugger
# docker-compose.prod.yml (explicit for production)
services:
app:
build:
target: production
restart: always
deploy:
resources:
limits:
cpus: "1.0"
memory: 512M
```
```bash
# Development (auto-loads override)
docker compose up
# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
```
## Networking
### Service Discovery
Services in the same Compose network resolve by service name:
```
# From "app" container:
postgres://postgres:postgres@db:5432/app_dev # "db" resolves to the db container
redis://redis:6379/0 # "redis" resolves to the redis container
```
### Custom Networks
```yaml
services:
frontend:
networks:
- frontend-net
api:
networks:
- frontend-net
- backend-net
db:
networks:
- backend-net # Only reachable from api, not frontend
networks:
frontend-net:
backend-net:
```
### Exposing Only What's Needed
```yaml
services:
db:
ports:
- "127.0.0.1:5432:5432" # Only accessible from host, not network
# Omit ports entirely in production -- accessible only within Docker network
```
## Volume Strategies
```yaml
volumes:
# Named volume: persists across container restarts, managed by Docker
pgdata:
# Bind mount: maps host directory into container (for development)
# - ./src:/app/src
# Anonymous volume: preserves container-generated content from bind mount override
# - /app/node_modules
```
### Common Patterns
```yaml
services:
app:
volumes:
- .:/app # Source code (bind mount for hot reload)
- /app/node_modules # Protect container's node_modules from host
- /app/.next # Protect build cache
db:
volumes:
- pgdata:/var/lib/postgresql/data # Persistent data
- ./scripts/init.sql:/docker-entrypoint-initdb.d/init.sql # Init scripts
```
## Container Security
### Dockerfile Hardening
```dockerfile
# 1. Use specific tags (never :latest)
FROM node:22.12-alpine3.20
# 2. Run as non-root
RUN addgroup -g 1001 -S app && adduser -S app -u 1001
USER app
# 3. Drop capabilities (in compose)
# 4. Read-only root filesystem where possible
# 5. No secrets in image layers
```
### Compose Security
```yaml
services:
app:
security_opt:
- no-new-privileges:true
read_only: true
tmpfs:
- /tmp
- /app/.cache
cap_drop:
- ALL
cap_add:
- NET_BIND_SERVICE # Only if binding to ports < 1024
```
### Secret Management
```yaml
# GOOD: Use environment variables (injected at runtime)
services:
app:
env_file:
- .env # Never commit .env to git
environment:
- API_KEY # Inherits from host environment
# GOOD: Docker secrets (Swarm mode)
secrets:
db_password:
file: ./secrets/db_password.txt
services:
db:
secrets:
- db_password
# BAD: Hardcoded in image
# ENV API_KEY=sk-proj-xxxxx # NEVER DO THIS
```
## .dockerignore
```
node_modules
.git
.env
.env.*
dist
coverage
*.log
.next
.cache
docker-compose*.yml
Dockerfile*
README.md
tests/
```
## Debugging
### Common Commands
```bash
# View logs
docker compose logs -f app # Follow app logs
docker compose logs --tail=50 db # Last 50 lines from db
# Execute commands in running container
docker compose exec app sh # Shell into app
docker compose exec db psql -U postgres # Connect to postgres
# Inspect
docker compose ps # Running services
docker compose top # Processes in each container
docker stats # Resource usage
# Rebuild
docker compose up --build # Rebuild images
docker compose build --no-cache app # Force full rebuild
# Clean up
docker compose down # Stop and remove containers
docker compose down -v # Also remove volumes (DESTRUCTIVE)
docker system prune # Remove unused images/containers
```
### Debugging Network Issues
```bash
# Check DNS resolution inside container
docker compose exec app nslookup db
# Check connectivity
docker compose exec app wget -qO- http://api:3000/health
# Inspect network
docker network ls
docker network inspect <project>_default
```
## Anti-Patterns
```
# BAD: Using docker compose in production without orchestration
# Use Kubernetes, ECS, or Docker Swarm for production multi-container workloads
# BAD: Storing data in containers without volumes
# Containers are ephemeral -- all data lost on restart without volumes
# BAD: Running as root
# Always create and use a non-root user
# BAD: Using :latest tag
# Pin to specific versions for reproducible builds
# BAD: One giant container with all services
# Separate concerns: one process per container
# BAD: Putting secrets in docker-compose.yml
# Use .env files (gitignored) or Docker secrets
```

325
skills/e2e-testing/SKILL.md Normal file
View File

@@ -0,0 +1,325 @@
---
name: e2e-testing
description: Playwright E2E testing patterns, Page Object Model, configuration, CI/CD integration, artifact management, and flaky test strategies.
---
# E2E Testing Patterns
Comprehensive Playwright patterns for building stable, fast, and maintainable E2E test suites.
## Test File Organization
```
tests/
├── e2e/
│ ├── auth/
│ │ ├── login.spec.ts
│ │ ├── logout.spec.ts
│ │ └── register.spec.ts
│ ├── features/
│ │ ├── browse.spec.ts
│ │ ├── search.spec.ts
│ │ └── create.spec.ts
│ └── api/
│ └── endpoints.spec.ts
├── fixtures/
│ ├── auth.ts
│ └── data.ts
└── playwright.config.ts
```
## Page Object Model (POM)
```typescript
import { Page, Locator } from '@playwright/test'
export class ItemsPage {
readonly page: Page
readonly searchInput: Locator
readonly itemCards: Locator
readonly createButton: Locator
constructor(page: Page) {
this.page = page
this.searchInput = page.locator('[data-testid="search-input"]')
this.itemCards = page.locator('[data-testid="item-card"]')
this.createButton = page.locator('[data-testid="create-btn"]')
}
async goto() {
await this.page.goto('/items')
await this.page.waitForLoadState('networkidle')
}
async search(query: string) {
await this.searchInput.fill(query)
await this.page.waitForResponse(resp => resp.url().includes('/api/search'))
await this.page.waitForLoadState('networkidle')
}
async getItemCount() {
return await this.itemCards.count()
}
}
```
## Test Structure
```typescript
import { test, expect } from '@playwright/test'
import { ItemsPage } from '../../pages/ItemsPage'
test.describe('Item Search', () => {
let itemsPage: ItemsPage
test.beforeEach(async ({ page }) => {
itemsPage = new ItemsPage(page)
await itemsPage.goto()
})
test('should search by keyword', async ({ page }) => {
await itemsPage.search('test')
const count = await itemsPage.getItemCount()
expect(count).toBeGreaterThan(0)
await expect(itemsPage.itemCards.first()).toContainText(/test/i)
await page.screenshot({ path: 'artifacts/search-results.png' })
})
test('should handle no results', async ({ page }) => {
await itemsPage.search('xyznonexistent123')
await expect(page.locator('[data-testid="no-results"]')).toBeVisible()
expect(await itemsPage.getItemCount()).toBe(0)
})
})
```
## Playwright Configuration
```typescript
import { defineConfig, devices } from '@playwright/test'
export default defineConfig({
testDir: './tests/e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: [
['html', { outputFolder: 'playwright-report' }],
['junit', { outputFile: 'playwright-results.xml' }],
['json', { outputFile: 'playwright-results.json' }]
],
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
actionTimeout: 10000,
navigationTimeout: 30000,
},
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
{ name: 'mobile-chrome', use: { ...devices['Pixel 5'] } },
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
timeout: 120000,
},
})
```
## Flaky Test Patterns
### Quarantine
```typescript
test('flaky: complex search', async ({ page }) => {
test.fixme(true, 'Flaky - Issue #123')
// test code...
})
test('conditional skip', async ({ page }) => {
test.skip(process.env.CI, 'Flaky in CI - Issue #123')
// test code...
})
```
### Identify Flakiness
```bash
npx playwright test tests/search.spec.ts --repeat-each=10
npx playwright test tests/search.spec.ts --retries=3
```
### Common Causes & Fixes
**Race conditions:**
```typescript
// Bad: assumes element is ready
await page.click('[data-testid="button"]')
// Good: auto-wait locator
await page.locator('[data-testid="button"]').click()
```
**Network timing:**
```typescript
// Bad: arbitrary timeout
await page.waitForTimeout(5000)
// Good: wait for specific condition
await page.waitForResponse(resp => resp.url().includes('/api/data'))
```
**Animation timing:**
```typescript
// Bad: click during animation
await page.click('[data-testid="menu-item"]')
// Good: wait for stability
await page.locator('[data-testid="menu-item"]').waitFor({ state: 'visible' })
await page.waitForLoadState('networkidle')
await page.locator('[data-testid="menu-item"]').click()
```
## Artifact Management
### Screenshots
```typescript
await page.screenshot({ path: 'artifacts/after-login.png' })
await page.screenshot({ path: 'artifacts/full-page.png', fullPage: true })
await page.locator('[data-testid="chart"]').screenshot({ path: 'artifacts/chart.png' })
```
### Traces
```typescript
await browser.startTracing(page, {
path: 'artifacts/trace.json',
screenshots: true,
snapshots: true,
})
// ... test actions ...
await browser.stopTracing()
```
### Video
```typescript
// In playwright.config.ts
use: {
video: 'retain-on-failure',
videosPath: 'artifacts/videos/'
}
```
## CI/CD Integration
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npx playwright install --with-deps
- run: npx playwright test
env:
BASE_URL: ${{ vars.STAGING_URL }}
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: playwright-report/
retention-days: 30
```
## Test Report Template
```markdown
# E2E Test Report
**Date:** YYYY-MM-DD HH:MM
**Duration:** Xm Ys
**Status:** PASSING / FAILING
## Summary
- Total: X | Passed: Y (Z%) | Failed: A | Flaky: B | Skipped: C
## Failed Tests
### test-name
**File:** `tests/e2e/feature.spec.ts:45`
**Error:** Expected element to be visible
**Screenshot:** artifacts/failed.png
**Recommended Fix:** [description]
## Artifacts
- HTML Report: playwright-report/index.html
- Screenshots: artifacts/*.png
- Videos: artifacts/videos/*.webm
- Traces: artifacts/*.zip
```
## Wallet / Web3 Testing
```typescript
test('wallet connection', async ({ page, context }) => {
// Mock wallet provider
await context.addInitScript(() => {
window.ethereum = {
isMetaMask: true,
request: async ({ method }) => {
if (method === 'eth_requestAccounts')
return ['0x1234567890123456789012345678901234567890']
if (method === 'eth_chainId') return '0x1'
}
}
})
await page.goto('/')
await page.locator('[data-testid="connect-wallet"]').click()
await expect(page.locator('[data-testid="wallet-address"]')).toContainText('0x1234')
})
```
## Financial / Critical Flow Testing
```typescript
test('trade execution', async ({ page }) => {
// Skip on production — real money
test.skip(process.env.NODE_ENV === 'production', 'Skip on production')
await page.goto('/markets/test-market')
await page.locator('[data-testid="position-yes"]').click()
await page.locator('[data-testid="trade-amount"]').fill('1.0')
// Verify preview
const preview = page.locator('[data-testid="trade-preview"]')
await expect(preview).toContainText('1.0')
// Confirm and wait for blockchain
await page.locator('[data-testid="confirm-trade"]').click()
await page.waitForResponse(
resp => resp.url().includes('/api/trade') && resp.status() === 200,
{ timeout: 30000 }
)
await expect(page.locator('[data-testid="trade-success"]')).toBeVisible()
})
```

235
skills/eval-harness/SKILL.md Normal file
View File

@@ -0,0 +1,235 @@
---
name: eval-harness
description: Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
tools: Read, Write, Edit, Bash, Grep, Glob
---
# Eval Harness Skill
A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.
## When to Activate
- Setting up eval-driven development (EDD) for AI-assisted workflows
- Defining pass/fail criteria for Claude Code task completion
- Measuring agent reliability with pass@k metrics
- Creating regression test suites for prompt or agent changes
- Benchmarking agent performance across model versions
## Philosophy
Eval-Driven Development treats evals as the "unit tests of AI development":
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
## Eval Types
### Capability Evals
Test if Claude can do something it couldn't before:
```markdown
[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
- [ ] Criterion 3
Expected Output: Description of expected result
```
### Regression Evals
Ensure changes don't break existing functionality:
```markdown
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
- existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)
```
## Grader Types
### 1. Code-Based Grader
Deterministic checks using code:
```bash
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"
```
### 2. Model-Based Grader
Use Claude to evaluate open-ended outputs:
```markdown
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?
Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]
```
### 3. Human Grader
Flag for manual review:
```markdown
[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH
```
## Metrics
### pass@k
"At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Typical target: pass@3 > 90%
### pass^k
"All k trials succeed"
- Higher bar for reliability
- pass^3: 3 consecutive successes
- Use for critical paths
## Eval Workflow
### 1. Define (Before Coding)
```markdown
## EVAL DEFINITION: feature-xyz
### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely
### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact
### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals
```
### 2. Implement
Write code to pass the defined evals.
### 3. Evaluate
```bash
# Run capability evals
[Run each capability eval, record PASS/FAIL]
# Run regression evals
npm test -- --testPathPattern="existing"
# Generate report
```
### 4. Report
```markdown
EVAL REPORT: feature-xyz
========================
Capability Evals:
create-user: PASS (pass@1)
validate-email: PASS (pass@2)
hash-password: PASS (pass@1)
Overall: 3/3 passed
Regression Evals:
login-flow: PASS
session-mgmt: PASS
logout-flow: PASS
Overall: 3/3 passed
Metrics:
pass@1: 67% (2/3)
pass@3: 100% (3/3)
Status: READY FOR REVIEW
```
## Integration Patterns
### Pre-Implementation
```
/eval define feature-name
```
Creates eval definition file at `.claude/evals/feature-name.md`
### During Implementation
```
/eval check feature-name
```
Runs current evals and reports status
### Post-Implementation
```
/eval report feature-name
```
Generates full eval report
## Eval Storage
Store evals in project:
```
.claude/
evals/
feature-xyz.md # Eval definition
feature-xyz.log # Eval run history
baseline.json # Regression baselines
```
## Best Practices
1. **Define evals BEFORE coding** - Forces clear thinking about success criteria
2. **Run evals frequently** - Catch regressions early
3. **Track pass@k over time** - Monitor reliability trends
4. **Use code graders when possible** - Deterministic > probabilistic
5. **Human review for security** - Never fully automate security checks
6. **Keep evals fast** - Slow evals don't get run
7. **Version evals with code** - Evals are first-class artifacts
## Example: Adding Authentication
```markdown
## EVAL: add-authentication
### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session
Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible
### Phase 2: Implement (varies)
[Write code]
### Phase 3: Evaluate
Run: /eval check add-authentication
### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT
```

641
skills/frontend-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,641 @@
---
name: frontend-patterns
description: Frontend development patterns for React, Next.js, state management, performance optimization, and UI best practices.
---
# Frontend Development Patterns
Modern frontend patterns for React, Next.js, and performant user interfaces.
## When to Activate
- Building React components (composition, props, rendering)
- Managing state (useState, useReducer, Zustand, Context)
- Implementing data fetching (SWR, React Query, server components)
- Optimizing performance (memoization, virtualization, code splitting)
- Working with forms (validation, controlled inputs, Zod schemas)
- Handling client-side routing and navigation
- Building accessible, responsive UI patterns
## Component Patterns
### Composition Over Inheritance
```typescript
// ✅ GOOD: Component composition
interface CardProps {
children: React.ReactNode
variant?: 'default' | 'outlined'
}
export function Card({ children, variant = 'default' }: CardProps) {
return <div className={`card card-${variant}`}>{children}</div>
}
export function CardHeader({ children }: { children: React.ReactNode }) {
return <div className="card-header">{children}</div>
}
export function CardBody({ children }: { children: React.ReactNode }) {
return <div className="card-body">{children}</div>
}
// Usage
<Card>
<CardHeader>Title</CardHeader>
<CardBody>Content</CardBody>
</Card>
```
### Compound Components
```typescript
interface TabsContextValue {
activeTab: string
setActiveTab: (tab: string) => void
}
const TabsContext = createContext<TabsContextValue | undefined>(undefined)
export function Tabs({ children, defaultTab }: {
children: React.ReactNode
defaultTab: string
}) {
const [activeTab, setActiveTab] = useState(defaultTab)
return (
<TabsContext.Provider value={{ activeTab, setActiveTab }}>
{children}
</TabsContext.Provider>
)
}
export function TabList({ children }: { children: React.ReactNode }) {
return <div className="tab-list">{children}</div>
}
export function Tab({ id, children }: { id: string, children: React.ReactNode }) {
const context = useContext(TabsContext)
if (!context) throw new Error('Tab must be used within Tabs')
return (
<button
className={context.activeTab === id ? 'active' : ''}
onClick={() => context.setActiveTab(id)}
>
{children}
</button>
)
}
// Usage
<Tabs defaultTab="overview">
<TabList>
<Tab id="overview">Overview</Tab>
<Tab id="details">Details</Tab>
</TabList>
</Tabs>
```
### Render Props Pattern
```typescript
interface DataLoaderProps<T> {
url: string
children: (data: T | null, loading: boolean, error: Error | null) => React.ReactNode
}
export function DataLoader<T>({ url, children }: DataLoaderProps<T>) {
const [data, setData] = useState<T | null>(null)
const [loading, setLoading] = useState(true)
const [error, setError] = useState<Error | null>(null)
useEffect(() => {
fetch(url)
.then(res => res.json())
.then(setData)
.catch(setError)
.finally(() => setLoading(false))
}, [url])
return <>{children(data, loading, error)}</>
}
// Usage
<DataLoader<Market[]> url="/api/markets">
{(markets, loading, error) => {
if (loading) return <Spinner />
if (error) return <Error error={error} />
return <MarketList markets={markets!} />
}}
</DataLoader>
```
## Custom Hooks Patterns
### State Management Hook
```typescript
export function useToggle(initialValue = false): [boolean, () => void] {
const [value, setValue] = useState(initialValue)
const toggle = useCallback(() => {
setValue(v => !v)
}, [])
return [value, toggle]
}
// Usage
const [isOpen, toggleOpen] = useToggle()
```
### Async Data Fetching Hook
```typescript
interface UseQueryOptions<T> {
onSuccess?: (data: T) => void
onError?: (error: Error) => void
enabled?: boolean
}
export function useQuery<T>(
key: string,
fetcher: () => Promise<T>,
options?: UseQueryOptions<T>
) {
const [data, setData] = useState<T | null>(null)
const [error, setError] = useState<Error | null>(null)
const [loading, setLoading] = useState(false)
const refetch = useCallback(async () => {
setLoading(true)
setError(null)
try {
const result = await fetcher()
setData(result)
options?.onSuccess?.(result)
} catch (err) {
const error = err as Error
setError(error)
options?.onError?.(error)
} finally {
setLoading(false)
}
}, [fetcher, options])
useEffect(() => {
if (options?.enabled !== false) {
refetch()
}
}, [key, refetch, options?.enabled])
return { data, error, loading, refetch }
}
// Usage
const { data: markets, loading, error, refetch } = useQuery(
'markets',
() => fetch('/api/markets').then(r => r.json()),
{
onSuccess: data => console.log('Fetched', data.length, 'markets'),
onError: err => console.error('Failed:', err)
}
)
```
### Debounce Hook
```typescript
export function useDebounce<T>(value: T, delay: number): T {
const [debouncedValue, setDebouncedValue] = useState<T>(value)
useEffect(() => {
const handler = setTimeout(() => {
setDebouncedValue(value)
}, delay)
return () => clearTimeout(handler)
}, [value, delay])
return debouncedValue
}
// Usage
const [searchQuery, setSearchQuery] = useState('')
const debouncedQuery = useDebounce(searchQuery, 500)
useEffect(() => {
if (debouncedQuery) {
performSearch(debouncedQuery)
}
}, [debouncedQuery])
```
## State Management Patterns
### Context + Reducer Pattern
```typescript
interface State {
markets: Market[]
selectedMarket: Market | null
loading: boolean
}
type Action =
| { type: 'SET_MARKETS'; payload: Market[] }
| { type: 'SELECT_MARKET'; payload: Market }
| { type: 'SET_LOADING'; payload: boolean }
function reducer(state: State, action: Action): State {
switch (action.type) {
case 'SET_MARKETS':
return { ...state, markets: action.payload }
case 'SELECT_MARKET':
return { ...state, selectedMarket: action.payload }
case 'SET_LOADING':
return { ...state, loading: action.payload }
default:
return state
}
}
const MarketContext = createContext<{
state: State
dispatch: Dispatch<Action>
} | undefined>(undefined)
export function MarketProvider({ children }: { children: React.ReactNode }) {
const [state, dispatch] = useReducer(reducer, {
markets: [],
selectedMarket: null,
loading: false
})
return (
<MarketContext.Provider value={{ state, dispatch }}>
{children}
</MarketContext.Provider>
)
}
export function useMarkets() {
const context = useContext(MarketContext)
if (!context) throw new Error('useMarkets must be used within MarketProvider')
return context
}
```
## Performance Optimization
### Memoization
```typescript
// ✅ useMemo for expensive computations
const sortedMarkets = useMemo(() => {
return markets.sort((a, b) => b.volume - a.volume)
}, [markets])
// ✅ useCallback for functions passed to children
const handleSearch = useCallback((query: string) => {
setSearchQuery(query)
}, [])
// ✅ React.memo for pure components
export const MarketCard = React.memo<MarketCardProps>(({ market }) => {
return (
<div className="market-card">
<h3>{market.name}</h3>
<p>{market.description}</p>
</div>
)
})
```
### Code Splitting & Lazy Loading
```typescript
import { lazy, Suspense } from 'react'
// ✅ Lazy load heavy components
const HeavyChart = lazy(() => import('./HeavyChart'))
const ThreeJsBackground = lazy(() => import('./ThreeJsBackground'))
export function Dashboard() {
return (
<div>
<Suspense fallback={<ChartSkeleton />}>
<HeavyChart data={data} />
</Suspense>
<Suspense fallback={null}>
<ThreeJsBackground />
</Suspense>
</div>
)
}
```
### Virtualization for Long Lists
```typescript
import { useVirtualizer } from '@tanstack/react-virtual'
export function VirtualMarketList({ markets }: { markets: Market[] }) {
const parentRef = useRef<HTMLDivElement>(null)
const virtualizer = useVirtualizer({
count: markets.length,
getScrollElement: () => parentRef.current,
estimateSize: () => 100, // Estimated row height
overscan: 5 // Extra items to render
})
return (
<div ref={parentRef} style={{ height: '600px', overflow: 'auto' }}>
<div
style={{
height: `${virtualizer.getTotalSize()}px`,
position: 'relative'
}}
>
{virtualizer.getVirtualItems().map(virtualRow => (
<div
key={virtualRow.index}
style={{
position: 'absolute',
top: 0,
left: 0,
width: '100%',
height: `${virtualRow.size}px`,
transform: `translateY(${virtualRow.start}px)`
}}
>
<MarketCard market={markets[virtualRow.index]} />
</div>
))}
</div>
</div>
)
}
```
## Form Handling Patterns
### Controlled Form with Validation
```typescript
interface FormData {
name: string
description: string
endDate: string
}
interface FormErrors {
name?: string
description?: string
endDate?: string
}
export function CreateMarketForm() {
const [formData, setFormData] = useState<FormData>({
name: '',
description: '',
endDate: ''
})
const [errors, setErrors] = useState<FormErrors>({})
const validate = (): boolean => {
const newErrors: FormErrors = {}
if (!formData.name.trim()) {
newErrors.name = 'Name is required'
} else if (formData.name.length > 200) {
newErrors.name = 'Name must be under 200 characters'
}
if (!formData.description.trim()) {
newErrors.description = 'Description is required'
}
if (!formData.endDate) {
newErrors.endDate = 'End date is required'
}
setErrors(newErrors)
return Object.keys(newErrors).length === 0
}
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault()
if (!validate()) return
try {
await createMarket(formData)
// Success handling
} catch (error) {
// Error handling
}
}
return (
<form onSubmit={handleSubmit}>
<input
value={formData.name}
onChange={e => setFormData(prev => ({ ...prev, name: e.target.value }))}
placeholder="Market name"
/>
{errors.name && <span className="error">{errors.name}</span>}
{/* Other fields */}
<button type="submit">Create Market</button>
</form>
)
}
```
## Error Boundary Pattern
```typescript
interface ErrorBoundaryState {
hasError: boolean
error: Error | null
}
export class ErrorBoundary extends React.Component<
{ children: React.ReactNode },
ErrorBoundaryState
> {
state: ErrorBoundaryState = {
hasError: false,
error: null
}
static getDerivedStateFromError(error: Error): ErrorBoundaryState {
return { hasError: true, error }
}
componentDidCatch(error: Error, errorInfo: React.ErrorInfo) {
console.error('Error boundary caught:', error, errorInfo)
}
render() {
if (this.state.hasError) {
return (
<div className="error-fallback">
<h2>Something went wrong</h2>
<p>{this.state.error?.message}</p>
<button onClick={() => this.setState({ hasError: false })}>
Try again
</button>
</div>
)
}
return this.props.children
}
}
// Usage
<ErrorBoundary>
<App />
</ErrorBoundary>
```
## Animation Patterns
### Framer Motion Animations
```typescript
import { motion, AnimatePresence } from 'framer-motion'
// ✅ List animations
export function AnimatedMarketList({ markets }: { markets: Market[] }) {
return (
<AnimatePresence>
{markets.map(market => (
<motion.div
key={market.id}
initial={{ opacity: 0, y: 20 }}
animate={{ opacity: 1, y: 0 }}
exit={{ opacity: 0, y: -20 }}
transition={{ duration: 0.3 }}
>
<MarketCard market={market} />
</motion.div>
))}
</AnimatePresence>
)
}
// ✅ Modal animations
export function Modal({ isOpen, onClose, children }: ModalProps) {
return (
<AnimatePresence>
{isOpen && (
<>
<motion.div
className="modal-overlay"
initial={{ opacity: 0 }}
animate={{ opacity: 1 }}
exit={{ opacity: 0 }}
onClick={onClose}
/>
<motion.div
className="modal-content"
initial={{ opacity: 0, scale: 0.9, y: 20 }}
animate={{ opacity: 1, scale: 1, y: 0 }}
exit={{ opacity: 0, scale: 0.9, y: 20 }}
>
{children}
</motion.div>
</>
)}
</AnimatePresence>
)
}
```
## Accessibility Patterns
### Keyboard Navigation
```typescript
export function Dropdown({ options, onSelect }: DropdownProps) {
const [isOpen, setIsOpen] = useState(false)
const [activeIndex, setActiveIndex] = useState(0)
const handleKeyDown = (e: React.KeyboardEvent) => {
switch (e.key) {
case 'ArrowDown':
e.preventDefault()
setActiveIndex(i => Math.min(i + 1, options.length - 1))
break
case 'ArrowUp':
e.preventDefault()
setActiveIndex(i => Math.max(i - 1, 0))
break
case 'Enter':
e.preventDefault()
onSelect(options[activeIndex])
setIsOpen(false)
break
case 'Escape':
setIsOpen(false)
break
}
}
return (
<div
role="combobox"
aria-expanded={isOpen}
aria-haspopup="listbox"
onKeyDown={handleKeyDown}
>
{/* Dropdown implementation */}
</div>
)
}
```
### Focus Management
```typescript
export function Modal({ isOpen, onClose, children }: ModalProps) {
const modalRef = useRef<HTMLDivElement>(null)
const previousFocusRef = useRef<HTMLElement | null>(null)
useEffect(() => {
if (isOpen) {
// Save currently focused element
previousFocusRef.current = document.activeElement as HTMLElement
// Focus modal
modalRef.current?.focus()
} else {
// Restore focus when closing
previousFocusRef.current?.focus()
}
}, [isOpen])
return isOpen ? (
<div
ref={modalRef}
role="dialog"
aria-modal="true"
tabIndex={-1}
onKeyDown={e => e.key === 'Escape' && onClose()}
>
{children}
</div>
) : null
}
```
**Remember**: Modern frontend patterns enable maintainable, performant user interfaces. Choose patterns that fit your project complexity.

53
skills/git.md Normal file
View File

@@ -0,0 +1,53 @@
# Git & Version Control Skill
## Git Configuration
```bash
git config --global user.name "ren"
git config --global user.email "ren@local"
git config --global init.defaultBranch main
```
## Common Commands
### Repository Operations
```bash
git init # Initialize repo
git clone <url> # Clone repo
git status # Check status
git add . # Stage all changes
git add file.js # Stage specific file
git commit -m "message" # Commit
git push origin main # Push to remote
git pull origin main # Pull from remote
```
### Branching
```bash
git branch # List branches
git branch new-feature # Create branch
git checkout new-feature # Switch branch
git checkout -b feature # Create and switch
git merge feature # Merge branch
git branch -d feature # Delete merged branch
```
### History & Diff
```bash
git log --oneline # Compact history
git diff # Show changes
git diff HEAD~1 # Compare with previous
git show <commit> # Show commit details
git reset --hard <commit> # Reset to commit
```
## Advanced
```bash
git stash # Stash changes
git stash pop # Apply stash
git cherry-pick <commit> # Pick specific commit
git rebase main # Rebase branch
```
## SSH Configuration
- SSH configured at: ~/.ssh/
- Public key can be added to GitHub/GitLab

673
skills/golang-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,673 @@
---
name: golang-patterns
description: Idiomatic Go patterns, best practices, and conventions for building robust, efficient, and maintainable Go applications.
---
# Go Development Patterns
Idiomatic Go patterns and best practices for building robust, efficient, and maintainable applications.
## When to Activate
- Writing new Go code
- Reviewing Go code
- Refactoring existing Go code
- Designing Go packages/modules
## Core Principles
### 1. Simplicity and Clarity
Go favors simplicity over cleverness. Code should be obvious and easy to read.
```go
// Good: Clear and direct
func GetUser(id string) (*User, error) {
user, err := db.FindUser(id)
if err != nil {
return nil, fmt.Errorf("get user %s: %w", id, err)
}
return user, nil
}
// Bad: Overly clever
func GetUser(id string) (*User, error) {
return func() (*User, error) {
if u, e := db.FindUser(id); e == nil {
return u, nil
} else {
return nil, e
}
}()
}
```
### 2. Make the Zero Value Useful
Design types so their zero value is immediately usable without initialization.
```go
// Good: Zero value is useful
type Counter struct {
mu sync.Mutex
count int // zero value is 0, ready to use
}
func (c *Counter) Inc() {
c.mu.Lock()
c.count++
c.mu.Unlock()
}
// Good: bytes.Buffer works with zero value
var buf bytes.Buffer
buf.WriteString("hello")
// Bad: Requires initialization
type BadCounter struct {
counts map[string]int // nil map will panic
}
```
### 3. Accept Interfaces, Return Structs
Functions should accept interface parameters and return concrete types.
```go
// Good: Accepts interface, returns concrete type
func ProcessData(r io.Reader) (*Result, error) {
data, err := io.ReadAll(r)
if err != nil {
return nil, err
}
return &Result{Data: data}, nil
}
// Bad: Returns interface (hides implementation details unnecessarily)
func ProcessData(r io.Reader) (io.Reader, error) {
// ...
}
```
## Error Handling Patterns
### Error Wrapping with Context
```go
// Good: Wrap errors with context
func LoadConfig(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("load config %s: %w", path, err)
}
var cfg Config
if err := json.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parse config %s: %w", path, err)
}
return &cfg, nil
}
```
### Custom Error Types
```go
// Define domain-specific errors
type ValidationError struct {
Field string
Message string
}
func (e *ValidationError) Error() string {
return fmt.Sprintf("validation failed on %s: %s", e.Field, e.Message)
}
// Sentinel errors for common cases
var (
ErrNotFound = errors.New("resource not found")
ErrUnauthorized = errors.New("unauthorized")
ErrInvalidInput = errors.New("invalid input")
)
```
### Error Checking with errors.Is and errors.As
```go
func HandleError(err error) {
// Check for specific error
if errors.Is(err, sql.ErrNoRows) {
log.Println("No records found")
return
}
// Check for error type
var validationErr *ValidationError
if errors.As(err, &validationErr) {
log.Printf("Validation error on field %s: %s",
validationErr.Field, validationErr.Message)
return
}
// Unknown error
log.Printf("Unexpected error: %v", err)
}
```
### Never Ignore Errors
```go
// Bad: Ignoring error with blank identifier
result, _ := doSomething()
// Good: Handle or explicitly document why it's safe to ignore
result, err := doSomething()
if err != nil {
return err
}
// Acceptable: When error truly doesn't matter (rare)
_ = writer.Close() // Best-effort cleanup, error logged elsewhere
```
## Concurrency Patterns
### Worker Pool
```go
func WorkerPool(jobs <-chan Job, results chan<- Result, numWorkers int) {
var wg sync.WaitGroup
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
results <- process(job)
}
}()
}
wg.Wait()
close(results)
}
```
### Context for Cancellation and Timeouts
```go
func FetchWithTimeout(ctx context.Context, url string) ([]byte, error) {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, fmt.Errorf("fetch %s: %w", url, err)
}
defer resp.Body.Close()
return io.ReadAll(resp.Body)
}
```
### Graceful Shutdown
```go
func GracefulShutdown(server *http.Server) {
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
log.Println("Shutting down server...")
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := server.Shutdown(ctx); err != nil {
log.Fatalf("Server forced to shutdown: %v", err)
}
log.Println("Server exited")
}
```
### errgroup for Coordinated Goroutines
```go
import "golang.org/x/sync/errgroup"
func FetchAll(ctx context.Context, urls []string) ([][]byte, error) {
g, ctx := errgroup.WithContext(ctx)
results := make([][]byte, len(urls))
for i, url := range urls {
i, url := i, url // Capture loop variables
g.Go(func() error {
data, err := FetchWithTimeout(ctx, url)
if err != nil {
return err
}
results[i] = data
return nil
})
}
if err := g.Wait(); err != nil {
return nil, err
}
return results, nil
}
```
### Avoiding Goroutine Leaks
```go
// Bad: Goroutine leak if context is cancelled
func leakyFetch(ctx context.Context, url string) <-chan []byte {
ch := make(chan []byte)
go func() {
data, _ := fetch(url)
ch <- data // Blocks forever if no receiver
}()
return ch
}
// Good: Properly handles cancellation
func safeFetch(ctx context.Context, url string) <-chan []byte {
ch := make(chan []byte, 1) // Buffered channel
go func() {
data, err := fetch(url)
if err != nil {
return
}
select {
case ch <- data:
case <-ctx.Done():
}
}()
return ch
}
```
## Interface Design
### Small, Focused Interfaces
```go
// Good: Single-method interfaces
type Reader interface {
Read(p []byte) (n int, err error)
}
type Writer interface {
Write(p []byte) (n int, err error)
}
type Closer interface {
Close() error
}
// Compose interfaces as needed
type ReadWriteCloser interface {
Reader
Writer
Closer
}
```
### Define Interfaces Where They're Used
```go
// In the consumer package, not the provider
package service
// UserStore defines what this service needs
type UserStore interface {
GetUser(id string) (*User, error)
SaveUser(user *User) error
}
type Service struct {
store UserStore
}
// Concrete implementation can be in another package
// It doesn't need to know about this interface
```
### Optional Behavior with Type Assertions
```go
type Flusher interface {
Flush() error
}
func WriteAndFlush(w io.Writer, data []byte) error {
if _, err := w.Write(data); err != nil {
return err
}
// Flush if supported
if f, ok := w.(Flusher); ok {
return f.Flush()
}
return nil
}
```
## Package Organization
### Standard Project Layout
```text
myproject/
├── cmd/
│ └── myapp/
│ └── main.go # Entry point
├── internal/
│ ├── handler/ # HTTP handlers
│ ├── service/ # Business logic
│ ├── repository/ # Data access
│ └── config/ # Configuration
├── pkg/
│ └── client/ # Public API client
├── api/
│ └── v1/ # API definitions (proto, OpenAPI)
├── testdata/ # Test fixtures
├── go.mod
├── go.sum
└── Makefile
```
### Package Naming
```go
// Good: Short, lowercase, no underscores
package http
package json
package user
// Bad: Verbose, mixed case, or redundant
package httpHandler
package json_parser
package userService // Redundant 'Service' suffix
```
### Avoid Package-Level State
```go
// Bad: Global mutable state
var db *sql.DB
func init() {
db, _ = sql.Open("postgres", os.Getenv("DATABASE_URL"))
}
// Good: Dependency injection
type Server struct {
db *sql.DB
}
func NewServer(db *sql.DB) *Server {
return &Server{db: db}
}
```
## Struct Design
### Functional Options Pattern
```go
type Server struct {
addr string
timeout time.Duration
logger *log.Logger
}
type Option func(*Server)
func WithTimeout(d time.Duration) Option {
return func(s *Server) {
s.timeout = d
}
}
func WithLogger(l *log.Logger) Option {
return func(s *Server) {
s.logger = l
}
}
func NewServer(addr string, opts ...Option) *Server {
s := &Server{
addr: addr,
timeout: 30 * time.Second, // default
logger: log.Default(), // default
}
for _, opt := range opts {
opt(s)
}
return s
}
// Usage
server := NewServer(":8080",
WithTimeout(60*time.Second),
WithLogger(customLogger),
)
```
### Embedding for Composition
```go
type Logger struct {
prefix string
}
func (l *Logger) Log(msg string) {
fmt.Printf("[%s] %s\n", l.prefix, msg)
}
type Server struct {
*Logger // Embedding - Server gets Log method
addr string
}
func NewServer(addr string) *Server {
return &Server{
Logger: &Logger{prefix: "SERVER"},
addr: addr,
}
}
// Usage
s := NewServer(":8080")
s.Log("Starting...") // Calls embedded Logger.Log
```
## Memory and Performance
### Preallocate Slices When Size is Known
```go
// Bad: Grows slice multiple times
func processItems(items []Item) []Result {
var results []Result
for _, item := range items {
results = append(results, process(item))
}
return results
}
// Good: Single allocation
func processItems(items []Item) []Result {
results := make([]Result, 0, len(items))
for _, item := range items {
results = append(results, process(item))
}
return results
}
```
### Use sync.Pool for Frequent Allocations
```go
var bufferPool = sync.Pool{
New: func() interface{} {
return new(bytes.Buffer)
},
}
func ProcessRequest(data []byte) []byte {
buf := bufferPool.Get().(*bytes.Buffer)
defer func() {
buf.Reset()
bufferPool.Put(buf)
}()
buf.Write(data)
// Process...
return buf.Bytes()
}
```
### Avoid String Concatenation in Loops
```go
// Bad: Creates many string allocations
func join(parts []string) string {
var result string
for _, p := range parts {
result += p + ","
}
return result
}
// Good: Single allocation with strings.Builder
func join(parts []string) string {
var sb strings.Builder
for i, p := range parts {
if i > 0 {
sb.WriteString(",")
}
sb.WriteString(p)
}
return sb.String()
}
// Best: Use standard library
func join(parts []string) string {
return strings.Join(parts, ",")
}
```
## Go Tooling Integration
### Essential Commands
```bash
# Build and run
go build ./...
go run ./cmd/myapp
# Testing
go test ./...
go test -race ./...
go test -cover ./...
# Static analysis
go vet ./...
staticcheck ./...
golangci-lint run
# Module management
go mod tidy
go mod verify
# Formatting
gofmt -w .
goimports -w .
```
### Recommended Linter Configuration (.golangci.yml)
```yaml
linters:
enable:
- errcheck
- gosimple
- govet
- ineffassign
- staticcheck
- unused
- gofmt
- goimports
- misspell
- unconvert
- unparam
linters-settings:
errcheck:
check-type-assertions: true
govet:
check-shadowing: true
issues:
exclude-use-default: false
```
## Quick Reference: Go Idioms
| Idiom | Description |
|-------|-------------|
| Accept interfaces, return structs | Functions accept interface params, return concrete types |
| Errors are values | Treat errors as first-class values, not exceptions |
| Don't communicate by sharing memory | Use channels for coordination between goroutines |
| Make the zero value useful | Types should work without explicit initialization |
| A little copying is better than a little dependency | Avoid unnecessary external dependencies |
| Clear is better than clever | Prioritize readability over cleverness |
| gofmt is no one's favorite but everyone's friend | Always format with gofmt/goimports |
| Return early | Handle errors first, keep happy path unindented |
## Anti-Patterns to Avoid
```go
// Bad: Naked returns in long functions
func process() (result int, err error) {
// ... 50 lines ...
return // What is being returned?
}
// Bad: Using panic for control flow
func GetUser(id string) *User {
user, err := db.Find(id)
if err != nil {
panic(err) // Don't do this
}
return user
}
// Bad: Passing context in struct
type Request struct {
ctx context.Context // Context should be first param
ID string
}
// Good: Context as first parameter
func ProcessRequest(ctx context.Context, id string) error {
// ...
}
// Bad: Mixing value and pointer receivers
type Counter struct{ n int }
func (c Counter) Value() int { return c.n } // Value receiver
func (c *Counter) Increment() { c.n++ } // Pointer receiver
// Pick one style and be consistent
```
**Remember**: Go code should be boring in the best way - predictable, consistent, and easy to understand. When in doubt, keep it simple.

719
skills/golang-testing/SKILL.md Normal file
View File

@@ -0,0 +1,719 @@
---
name: golang-testing
description: Go testing patterns including table-driven tests, subtests, benchmarks, fuzzing, and test coverage. Follows TDD methodology with idiomatic Go practices.
---
# Go Testing Patterns
Comprehensive Go testing patterns for writing reliable, maintainable tests following TDD methodology.
## When to Activate
- Writing new Go functions or methods
- Adding test coverage to existing code
- Creating benchmarks for performance-critical code
- Implementing fuzz tests for input validation
- Following TDD workflow in Go projects
## TDD Workflow for Go
### The RED-GREEN-REFACTOR Cycle
```
RED → Write a failing test first
GREEN → Write minimal code to pass the test
REFACTOR → Improve code while keeping tests green
REPEAT → Continue with next requirement
```
### Step-by-Step TDD in Go
```go
// Step 1: Define the interface/signature
// calculator.go
package calculator
func Add(a, b int) int {
panic("not implemented") // Placeholder
}
// Step 2: Write failing test (RED)
// calculator_test.go
package calculator
import "testing"
func TestAdd(t *testing.T) {
got := Add(2, 3)
want := 5
if got != want {
t.Errorf("Add(2, 3) = %d; want %d", got, want)
}
}
// Step 3: Run test - verify FAIL
// $ go test
// --- FAIL: TestAdd (0.00s)
// panic: not implemented
// Step 4: Implement minimal code (GREEN)
func Add(a, b int) int {
return a + b
}
// Step 5: Run test - verify PASS
// $ go test
// PASS
// Step 6: Refactor if needed, verify tests still pass
```
## Table-Driven Tests
The standard pattern for Go tests. Enables comprehensive coverage with minimal code.
```go
func TestAdd(t *testing.T) {
tests := []struct {
name string
a, b int
expected int
}{
{"positive numbers", 2, 3, 5},
{"negative numbers", -1, -2, -3},
{"zero values", 0, 0, 0},
{"mixed signs", -1, 1, 0},
{"large numbers", 1000000, 2000000, 3000000},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := Add(tt.a, tt.b)
if got != tt.expected {
t.Errorf("Add(%d, %d) = %d; want %d",
tt.a, tt.b, got, tt.expected)
}
})
}
}
```
### Table-Driven Tests with Error Cases
```go
func TestParseConfig(t *testing.T) {
tests := []struct {
name string
input string
want *Config
wantErr bool
}{
{
name: "valid config",
input: `{"host": "localhost", "port": 8080}`,
want: &Config{Host: "localhost", Port: 8080},
},
{
name: "invalid JSON",
input: `{invalid}`,
wantErr: true,
},
{
name: "empty input",
input: "",
wantErr: true,
},
{
name: "minimal config",
input: `{}`,
want: &Config{}, // Zero value config
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ParseConfig(tt.input)
if tt.wantErr {
if err == nil {
t.Error("expected error, got nil")
}
return
}
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("got %+v; want %+v", got, tt.want)
}
})
}
}
```
## Subtests and Sub-benchmarks
### Organizing Related Tests
```go
func TestUser(t *testing.T) {
// Setup shared by all subtests
db := setupTestDB(t)
t.Run("Create", func(t *testing.T) {
user := &User{Name: "Alice"}
err := db.CreateUser(user)
if err != nil {
t.Fatalf("CreateUser failed: %v", err)
}
if user.ID == "" {
t.Error("expected user ID to be set")
}
})
t.Run("Get", func(t *testing.T) {
user, err := db.GetUser("alice-id")
if err != nil {
t.Fatalf("GetUser failed: %v", err)
}
if user.Name != "Alice" {
t.Errorf("got name %q; want %q", user.Name, "Alice")
}
})
t.Run("Update", func(t *testing.T) {
// ...
})
t.Run("Delete", func(t *testing.T) {
// ...
})
}
```
### Parallel Subtests
```go
func TestParallel(t *testing.T) {
tests := []struct {
name string
input string
}{
{"case1", "input1"},
{"case2", "input2"},
{"case3", "input3"},
}
for _, tt := range tests {
tt := tt // Capture range variable
t.Run(tt.name, func(t *testing.T) {
t.Parallel() // Run subtests in parallel
result := Process(tt.input)
// assertions...
_ = result
})
}
}
```
## Test Helpers
### Helper Functions
```go
func setupTestDB(t *testing.T) *sql.DB {
t.Helper() // Marks this as a helper function
db, err := sql.Open("sqlite3", ":memory:")
if err != nil {
t.Fatalf("failed to open database: %v", err)
}
// Cleanup when test finishes
t.Cleanup(func() {
db.Close()
})
// Run migrations
if _, err := db.Exec(schema); err != nil {
t.Fatalf("failed to create schema: %v", err)
}
return db
}
func assertNoError(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func assertEqual[T comparable](t *testing.T, got, want T) {
t.Helper()
if got != want {
t.Errorf("got %v; want %v", got, want)
}
}
```
### Temporary Files and Directories
```go
func TestFileProcessing(t *testing.T) {
// Create temp directory - automatically cleaned up
tmpDir := t.TempDir()
// Create test file
testFile := filepath.Join(tmpDir, "test.txt")
err := os.WriteFile(testFile, []byte("test content"), 0644)
if err != nil {
t.Fatalf("failed to create test file: %v", err)
}
// Run test
result, err := ProcessFile(testFile)
if err != nil {
t.Fatalf("ProcessFile failed: %v", err)
}
// Assert...
_ = result
}
```
## Golden Files
Testing against expected output files stored in `testdata/`.
```go
var update = flag.Bool("update", false, "update golden files")
func TestRender(t *testing.T) {
tests := []struct {
name string
input Template
}{
{"simple", Template{Name: "test"}},
{"complex", Template{Name: "test", Items: []string{"a", "b"}}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := Render(tt.input)
golden := filepath.Join("testdata", tt.name+".golden")
if *update {
// Update golden file: go test -update
err := os.WriteFile(golden, got, 0644)
if err != nil {
t.Fatalf("failed to update golden file: %v", err)
}
}
want, err := os.ReadFile(golden)
if err != nil {
t.Fatalf("failed to read golden file: %v", err)
}
if !bytes.Equal(got, want) {
t.Errorf("output mismatch:\ngot:\n%s\nwant:\n%s", got, want)
}
})
}
}
```
## Mocking with Interfaces
### Interface-Based Mocking
```go
// Define interface for dependencies
type UserRepository interface {
GetUser(id string) (*User, error)
SaveUser(user *User) error
}
// Production implementation
type PostgresUserRepository struct {
db *sql.DB
}
func (r *PostgresUserRepository) GetUser(id string) (*User, error) {
// Real database query
}
// Mock implementation for tests
type MockUserRepository struct {
GetUserFunc func(id string) (*User, error)
SaveUserFunc func(user *User) error
}
func (m *MockUserRepository) GetUser(id string) (*User, error) {
return m.GetUserFunc(id)
}
func (m *MockUserRepository) SaveUser(user *User) error {
return m.SaveUserFunc(user)
}
// Test using mock
func TestUserService(t *testing.T) {
mock := &MockUserRepository{
GetUserFunc: func(id string) (*User, error) {
if id == "123" {
return &User{ID: "123", Name: "Alice"}, nil
}
return nil, ErrNotFound
},
}
service := NewUserService(mock)
user, err := service.GetUserProfile("123")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if user.Name != "Alice" {
t.Errorf("got name %q; want %q", user.Name, "Alice")
}
}
```
## Benchmarks
### Basic Benchmarks
```go
func BenchmarkProcess(b *testing.B) {
data := generateTestData(1000)
b.ResetTimer() // Don't count setup time
for i := 0; i < b.N; i++ {
Process(data)
}
}
// Run: go test -bench=BenchmarkProcess -benchmem
// Output: BenchmarkProcess-8 10000 105234 ns/op 4096 B/op 10 allocs/op
```
### Benchmark with Different Sizes
```go
func BenchmarkSort(b *testing.B) {
sizes := []int{100, 1000, 10000, 100000}
for _, size := range sizes {
b.Run(fmt.Sprintf("size=%d", size), func(b *testing.B) {
data := generateRandomSlice(size)
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Make a copy to avoid sorting already sorted data
tmp := make([]int, len(data))
copy(tmp, data)
sort.Ints(tmp)
}
})
}
}
```
### Memory Allocation Benchmarks
```go
func BenchmarkStringConcat(b *testing.B) {
parts := []string{"hello", "world", "foo", "bar", "baz"}
b.Run("plus", func(b *testing.B) {
for i := 0; i < b.N; i++ {
var s string
for _, p := range parts {
s += p
}
_ = s
}
})
b.Run("builder", func(b *testing.B) {
for i := 0; i < b.N; i++ {
var sb strings.Builder
for _, p := range parts {
sb.WriteString(p)
}
_ = sb.String()
}
})
b.Run("join", func(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = strings.Join(parts, "")
}
})
}
```
## Fuzzing (Go 1.18+)
### Basic Fuzz Test
```go
func FuzzParseJSON(f *testing.F) {
// Add seed corpus
f.Add(`{"name": "test"}`)
f.Add(`{"count": 123}`)
f.Add(`[]`)
f.Add(`""`)
f.Fuzz(func(t *testing.T, input string) {
var result map[string]interface{}
err := json.Unmarshal([]byte(input), &result)
if err != nil {
// Invalid JSON is expected for random input
return
}
// If parsing succeeded, re-encoding should work
_, err = json.Marshal(result)
if err != nil {
t.Errorf("Marshal failed after successful Unmarshal: %v", err)
}
})
}
// Run: go test -fuzz=FuzzParseJSON -fuzztime=30s
```
### Fuzz Test with Multiple Inputs
```go
func FuzzCompare(f *testing.F) {
f.Add("hello", "world")
f.Add("", "")
f.Add("abc", "abc")
f.Fuzz(func(t *testing.T, a, b string) {
result := Compare(a, b)
// Property: Compare(a, a) should always equal 0
if a == b && result != 0 {
t.Errorf("Compare(%q, %q) = %d; want 0", a, b, result)
}
// Property: Compare(a, b) and Compare(b, a) should have opposite signs
reverse := Compare(b, a)
if (result > 0 && reverse >= 0) || (result < 0 && reverse <= 0) {
if result != 0 || reverse != 0 {
t.Errorf("Compare(%q, %q) = %d, Compare(%q, %q) = %d; inconsistent",
a, b, result, b, a, reverse)
}
}
})
}
```
## Test Coverage
### Running Coverage
```bash
# Basic coverage
go test -cover ./...
# Generate coverage profile
go test -coverprofile=coverage.out ./...
# View coverage in browser
go tool cover -html=coverage.out
# View coverage by function
go tool cover -func=coverage.out
# Coverage with race detection
go test -race -coverprofile=coverage.out ./...
```
### Coverage Targets
| Code Type | Target |
|-----------|--------|
| Critical business logic | 100% |
| Public APIs | 90%+ |
| General code | 80%+ |
| Generated code | Exclude |
### Excluding Generated Code from Coverage
```go
//go:generate mockgen -source=interface.go -destination=mock_interface.go
// In coverage profile, exclude with build tags:
// go test -cover -tags=!generate ./...
```
## HTTP Handler Testing
```go
func TestHealthHandler(t *testing.T) {
// Create request
req := httptest.NewRequest(http.MethodGet, "/health", nil)
w := httptest.NewRecorder()
// Call handler
HealthHandler(w, req)
// Check response
resp := w.Result()
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Errorf("got status %d; want %d", resp.StatusCode, http.StatusOK)
}
body, _ := io.ReadAll(resp.Body)
if string(body) != "OK" {
t.Errorf("got body %q; want %q", body, "OK")
}
}
func TestAPIHandler(t *testing.T) {
tests := []struct {
name string
method string
path string
body string
wantStatus int
wantBody string
}{
{
name: "get user",
method: http.MethodGet,
path: "/users/123",
wantStatus: http.StatusOK,
wantBody: `{"id":"123","name":"Alice"}`,
},
{
name: "not found",
method: http.MethodGet,
path: "/users/999",
wantStatus: http.StatusNotFound,
},
{
name: "create user",
method: http.MethodPost,
path: "/users",
body: `{"name":"Bob"}`,
wantStatus: http.StatusCreated,
},
}
handler := NewAPIHandler()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var body io.Reader
if tt.body != "" {
body = strings.NewReader(tt.body)
}
req := httptest.NewRequest(tt.method, tt.path, body)
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
if w.Code != tt.wantStatus {
t.Errorf("got status %d; want %d", w.Code, tt.wantStatus)
}
if tt.wantBody != "" && w.Body.String() != tt.wantBody {
t.Errorf("got body %q; want %q", w.Body.String(), tt.wantBody)
}
})
}
}
```
## Testing Commands
```bash
# Run all tests
go test ./...
# Run tests with verbose output
go test -v ./...
# Run specific test
go test -run TestAdd ./...
# Run tests matching pattern
go test -run "TestUser/Create" ./...
# Run tests with race detector
go test -race ./...
# Run tests with coverage
go test -cover -coverprofile=coverage.out ./...
# Run short tests only
go test -short ./...
# Run tests with timeout
go test -timeout 30s ./...
# Run benchmarks
go test -bench=. -benchmem ./...
# Run fuzzing
go test -fuzz=FuzzParse -fuzztime=30s ./...
# Count test runs (for flaky test detection)
go test -count=10 ./...
```
## Best Practices
**DO:**
- Write tests FIRST (TDD)
- Use table-driven tests for comprehensive coverage
- Test behavior, not implementation
- Use `t.Helper()` in helper functions
- Use `t.Parallel()` for independent tests
- Clean up resources with `t.Cleanup()`
- Use meaningful test names that describe the scenario
**DON'T:**
- Test private functions directly (test through public API)
- Use `time.Sleep()` in tests (use channels or conditions)
- Ignore flaky tests (fix or remove them)
- Mock everything (prefer integration tests when possible)
- Skip error path testing
## Integration with CI/CD
```yaml
# GitHub Actions example
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.22'
- name: Run tests
run: go test -race -coverprofile=coverage.out ./...
- name: Check coverage
run: |
go tool cover -func=coverage.out | grep total | awk '{print $3}' | \
awk -F'%' '{if ($1 < 80) exit 1}'
```
**Remember**: Tests are documentation. They show how your code is meant to be used. Write them clearly and keep them up to date.

210
skills/iterative-retrieval/SKILL.md Normal file
View File

@@ -0,0 +1,210 @@
---
name: iterative-retrieval
description: Pattern for progressively refining context retrieval to solve the subagent context problem
---
# Iterative Retrieval Pattern
Solves the "context problem" in multi-agent workflows where subagents don't know what context they need until they start working.
## When to Activate
- Spawning subagents that need codebase context they cannot predict upfront
- Building multi-agent workflows where context is progressively refined
- Encountering "context too large" or "missing context" failures in agent tasks
- Designing RAG-like retrieval pipelines for code exploration
- Optimizing token usage in agent orchestration
## The Problem
Subagents are spawned with limited context. They don't know:
- Which files contain relevant code
- What patterns exist in the codebase
- What terminology the project uses
Standard approaches fail:
- **Send everything**: Exceeds context limits
- **Send nothing**: Agent lacks critical information
- **Guess what's needed**: Often wrong
## The Solution: Iterative Retrieval
A 4-phase loop that progressively refines context:
```
┌─────────────────────────────────────────────┐
│ │
│ ┌──────────┐ ┌──────────┐ │
│ │ DISPATCH │─────▶│ EVALUATE │ │
│ └──────────┘ └──────────┘ │
│ ▲ │ │
│ │ ▼ │
│ ┌──────────┐ ┌──────────┐ │
│ │ LOOP │◀─────│ REFINE │ │
│ └──────────┘ └──────────┘ │
│ │
│ Max 3 cycles, then proceed │
└─────────────────────────────────────────────┘
```
### Phase 1: DISPATCH
Initial broad query to gather candidate files:
```javascript
// Start with high-level intent
const initialQuery = {
patterns: ['src/**/*.ts', 'lib/**/*.ts'],
keywords: ['authentication', 'user', 'session'],
excludes: ['*.test.ts', '*.spec.ts']
};
// Dispatch to retrieval agent
const candidates = await retrieveFiles(initialQuery);
```
### Phase 2: EVALUATE
Assess retrieved content for relevance:
```javascript
function evaluateRelevance(files, task) {
return files.map(file => ({
path: file.path,
relevance: scoreRelevance(file.content, task),
reason: explainRelevance(file.content, task),
missingContext: identifyGaps(file.content, task)
}));
}
```
Scoring criteria:
- **High (0.8-1.0)**: Directly implements target functionality
- **Medium (0.5-0.7)**: Contains related patterns or types
- **Low (0.2-0.4)**: Tangentially related
- **None (0-0.2)**: Not relevant, exclude
### Phase 3: REFINE
Update search criteria based on evaluation:
```javascript
function refineQuery(evaluation, previousQuery) {
return {
// Add new patterns discovered in high-relevance files
patterns: [...previousQuery.patterns, ...extractPatterns(evaluation)],
// Add terminology found in codebase
keywords: [...previousQuery.keywords, ...extractKeywords(evaluation)],
// Exclude confirmed irrelevant paths
excludes: [...previousQuery.excludes, ...evaluation
.filter(e => e.relevance < 0.2)
.map(e => e.path)
],
// Target specific gaps
focusAreas: evaluation
.flatMap(e => e.missingContext)
.filter(unique)
};
}
```
### Phase 4: LOOP
Repeat with refined criteria (max 3 cycles):
```javascript
async function iterativeRetrieve(task, maxCycles = 3) {
let query = createInitialQuery(task);
let bestContext = [];
for (let cycle = 0; cycle < maxCycles; cycle++) {
const candidates = await retrieveFiles(query);
const evaluation = evaluateRelevance(candidates, task);
// Check if we have sufficient context
const highRelevance = evaluation.filter(e => e.relevance >= 0.7);
if (highRelevance.length >= 3 && !hasCriticalGaps(evaluation)) {
return highRelevance;
}
// Refine and continue
query = refineQuery(evaluation, query);
bestContext = mergeContext(bestContext, highRelevance);
}
return bestContext;
}
```
## Practical Examples
### Example 1: Bug Fix Context
```
Task: "Fix the authentication token expiry bug"
Cycle 1:
DISPATCH: Search for "token", "auth", "expiry" in src/**
EVALUATE: Found auth.ts (0.9), tokens.ts (0.8), user.ts (0.3)
REFINE: Add "refresh", "jwt" keywords; exclude user.ts
Cycle 2:
DISPATCH: Search refined terms
EVALUATE: Found session-manager.ts (0.95), jwt-utils.ts (0.85)
REFINE: Sufficient context (2 high-relevance files)
Result: auth.ts, tokens.ts, session-manager.ts, jwt-utils.ts
```
### Example 2: Feature Implementation
```
Task: "Add rate limiting to API endpoints"
Cycle 1:
DISPATCH: Search "rate", "limit", "api" in routes/**
EVALUATE: No matches - codebase uses "throttle" terminology
REFINE: Add "throttle", "middleware" keywords
Cycle 2:
DISPATCH: Search refined terms
EVALUATE: Found throttle.ts (0.9), middleware/index.ts (0.7)
REFINE: Need router patterns
Cycle 3:
DISPATCH: Search "router", "express" patterns
EVALUATE: Found router-setup.ts (0.8)
REFINE: Sufficient context
Result: throttle.ts, middleware/index.ts, router-setup.ts
```
## Integration with Agents
Use in agent prompts:
```markdown
When retrieving context for this task:
1. Start with broad keyword search
2. Evaluate each file's relevance (0-1 scale)
3. Identify what context is still missing
4. Refine search criteria and repeat (max 3 cycles)
5. Return files with relevance >= 0.7
```
## Best Practices
1. **Start broad, narrow progressively** - Don't over-specify initial queries
2. **Learn codebase terminology** - First cycle often reveals naming conventions
3. **Track what's missing** - Explicit gap identification drives refinement
4. **Stop at "good enough"** - 3 high-relevance files beats 10 mediocre ones
5. **Exclude confidently** - Low-relevance files won't become relevant
## Related
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) - Subagent orchestration section
- `continuous-learning` skill - For patterns that improve over time
- Agent definitions in `~/.claude/agents/`

146
skills/java-coding-standards/SKILL.md Normal file
View File

@@ -0,0 +1,146 @@
---
name: java-coding-standards
description: "Java coding standards for Spring Boot services: naming, immutability, Optional usage, streams, exceptions, generics, and project layout."
---
# Java Coding Standards
Standards for readable, maintainable Java (17+) code in Spring Boot services.
## When to Activate
- Writing or reviewing Java code in Spring Boot projects
- Enforcing naming, immutability, or exception handling conventions
- Working with records, sealed classes, or pattern matching (Java 17+)
- Reviewing use of Optional, streams, or generics
- Structuring packages and project layout
## Core Principles
- Prefer clarity over cleverness
- Immutable by default; minimize shared mutable state
- Fail fast with meaningful exceptions
- Consistent naming and package structure
## Naming
```java
// ✅ Classes/Records: PascalCase
public class MarketService {}
public record Money(BigDecimal amount, Currency currency) {}
// ✅ Methods/fields: camelCase
private final MarketRepository marketRepository;
public Market findBySlug(String slug) {}
// ✅ Constants: UPPER_SNAKE_CASE
private static final int MAX_PAGE_SIZE = 100;
```
## Immutability
```java
// ✅ Favor records and final fields
public record MarketDto(Long id, String name, MarketStatus status) {}
public class Market {
private final Long id;
private final String name;
// getters only, no setters
}
```
## Optional Usage
```java
// ✅ Return Optional from find* methods
Optional<Market> market = marketRepository.findBySlug(slug);
// ✅ Map/flatMap instead of get()
return market
.map(MarketResponse::from)
.orElseThrow(() -> new EntityNotFoundException("Market not found"));
```
## Streams Best Practices
```java
// ✅ Use streams for transformations, keep pipelines short
List<String> names = markets.stream()
.map(Market::name)
.filter(Objects::nonNull)
.toList();
// ❌ Avoid complex nested streams; prefer loops for clarity
```
## Exceptions
- Use unchecked exceptions for domain errors; wrap technical exceptions with context
- Create domain-specific exceptions (e.g., `MarketNotFoundException`)
- Avoid broad `catch (Exception ex)` unless rethrowing/logging centrally
```java
throw new MarketNotFoundException(slug);
```
## Generics and Type Safety
- Avoid raw types; declare generic parameters
- Prefer bounded generics for reusable utilities
```java
public <T extends Identifiable> Map<Long, T> indexById(Collection<T> items) { ... }
```
## Project Structure (Maven/Gradle)
```
src/main/java/com/example/app/
config/
controller/
service/
repository/
domain/
dto/
util/
src/main/resources/
application.yml
src/test/java/... (mirrors main)
```
## Formatting and Style
- Use 2 or 4 spaces consistently (project standard)
- One public top-level type per file
- Keep methods short and focused; extract helpers
- Order members: constants, fields, constructors, public methods, protected, private
## Code Smells to Avoid
- Long parameter lists → use DTO/builders
- Deep nesting → early returns
- Magic numbers → named constants
- Static mutable state → prefer dependency injection
- Silent catch blocks → log and act or rethrow
## Logging
```java
private static final Logger log = LoggerFactory.getLogger(MarketService.class);
log.info("fetch_market slug={}", slug);
log.error("failed_fetch_market slug={}", slug, ex);
```
## Null Handling
- Accept `@Nullable` only when unavoidable; otherwise use `@NonNull`
- Use Bean Validation (`@NotNull`, `@NotBlank`) on inputs
## Testing Expectations
- JUnit 5 + AssertJ for fluent assertions
- Mockito for mocking; avoid partial mocks where possible
- Favor deterministic tests; no hidden sleeps
**Remember**: Keep code intentional, typed, and observable. Optimize for maintainability over micro-optimizations unless proven necessary.

150
skills/jpa-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,150 @@
---
name: jpa-patterns
description: JPA/Hibernate patterns for entity design, relationships, query optimization, transactions, auditing, indexing, pagination, and pooling in Spring Boot.
---
# JPA/Hibernate Patterns
Use for data modeling, repositories, and performance tuning in Spring Boot.
## When to Activate
- Designing JPA entities and table mappings
- Defining relationships (@OneToMany, @ManyToOne, @ManyToMany)
- Optimizing queries (N+1 prevention, fetch strategies, projections)
- Configuring transactions, auditing, or soft deletes
- Setting up pagination, sorting, or custom repository methods
- Tuning connection pooling (HikariCP) or second-level caching
## Entity Design
```java
@Entity
@Table(name = "markets", indexes = {
@Index(name = "idx_markets_slug", columnList = "slug", unique = true)
})
@EntityListeners(AuditingEntityListener.class)
public class MarketEntity {
@Id @GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@Column(nullable = false, length = 200)
private String name;
@Column(nullable = false, unique = true, length = 120)
private String slug;
@Enumerated(EnumType.STRING)
private MarketStatus status = MarketStatus.ACTIVE;
@CreatedDate private Instant createdAt;
@LastModifiedDate private Instant updatedAt;
}
```
Enable auditing:
```java
@Configuration
@EnableJpaAuditing
class JpaConfig {}
```
## Relationships and N+1 Prevention
```java
@OneToMany(mappedBy = "market", cascade = CascadeType.ALL, orphanRemoval = true)
private List<PositionEntity> positions = new ArrayList<>();
```
- Default to lazy loading; use `JOIN FETCH` in queries when needed
- Avoid `EAGER` on collections; use DTO projections for read paths
```java
@Query("select m from MarketEntity m left join fetch m.positions where m.id = :id")
Optional<MarketEntity> findWithPositions(@Param("id") Long id);
```
## Repository Patterns
```java
public interface MarketRepository extends JpaRepository<MarketEntity, Long> {
Optional<MarketEntity> findBySlug(String slug);
@Query("select m from MarketEntity m where m.status = :status")
Page<MarketEntity> findByStatus(@Param("status") MarketStatus status, Pageable pageable);
}
```
- Use projections for lightweight queries:
```java
public interface MarketSummary {
Long getId();
String getName();
MarketStatus getStatus();
}
Page<MarketSummary> findAllBy(Pageable pageable);
```
## Transactions
- Annotate service methods with `@Transactional`
- Use `@Transactional(readOnly = true)` for read paths to optimize
- Choose propagation carefully; avoid long-running transactions
```java
@Transactional
public Market updateStatus(Long id, MarketStatus status) {
MarketEntity entity = repo.findById(id)
.orElseThrow(() -> new EntityNotFoundException("Market"));
entity.setStatus(status);
return Market.from(entity);
}
```
## Pagination
```java
PageRequest page = PageRequest.of(pageNumber, pageSize, Sort.by("createdAt").descending());
Page<MarketEntity> markets = repo.findByStatus(MarketStatus.ACTIVE, page);
```
For cursor-like pagination, include `id > :lastId` in JPQL with ordering.
## Indexing and Performance
- Add indexes for common filters (`status`, `slug`, foreign keys)
- Use composite indexes matching query patterns (`status, created_at`)
- Avoid `select *`; project only needed columns
- Batch writes with `saveAll` and `hibernate.jdbc.batch_size`
## Connection Pooling (HikariCP)
Recommended properties:
```
spring.datasource.hikari.maximum-pool-size=20
spring.datasource.hikari.minimum-idle=5
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.validation-timeout=5000
```
For PostgreSQL LOB handling, add:
```
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
```
## Caching
- 1st-level cache is per EntityManager; avoid keeping entities across transactions
- For read-heavy entities, consider second-level cache cautiously; validate eviction strategy
## Migrations
- Use Flyway or Liquibase; never rely on Hibernate auto DDL in production
- Keep migrations idempotent and additive; avoid dropping columns without plan
## Testing Data Access
- Prefer `@DataJpaTest` with Testcontainers to mirror production
- Assert SQL efficiency using logs: set `logging.level.org.hibernate.SQL=DEBUG` and `logging.level.org.hibernate.orm.jdbc.bind=TRACE` for parameter values
**Remember**: Keep entities lean, queries intentional, and transactions short. Prevent N+1 with fetch strategies and projections, and index for your read/write paths.

64
skills/nodejs.md Normal file
View File

@@ -0,0 +1,64 @@
# Node.js & JavaScript Skill
## Installed Versions
- Node.js 20.x
- npm 10.x
- Claude Code CLI
## Common Commands
### Project Initialization
```bash
npm init -y # Initialize package.json
npm install express # Install dependency
npm install -D nodemon # Install dev dependency
npm install --save-exact # Install exact version
```
### Development
```bash
npm start # Start app
npm run dev # Start with hot reload
npm test # Run tests
npm run build # Build for production
npm outdated # Check for updates
npm update # Update dependencies
```
### Docker for Node
```bash
# Basic Dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
CMD ["npm", "start"]
```
## Project Templates
### Express API
```javascript
const express = require('express');
const app = express();
const port = 3000;
app.use(express.json());
app.get('/', (req, res) => res.send('Hello'));
app.listen(port, () => console.log(`Server on ${port}`));
```
### Fast Setup
```bash
npx create-next-app myapp # Next.js
npx create-react-app myapp # React
npx create-vue-app myapp # Vue
```
## Best Practices
- Use `npm ci` instead of `npm install` in CI/CD
- Use `package-lock.json` for reproducible builds
- Define scripts in package.json for common tasks
- Use environment variables with `dotenv`

165
skills/nutrient-document-processing/SKILL.md Normal file
View File

@@ -0,0 +1,165 @@
---
name: nutrient-document-processing
description: Process, convert, OCR, extract, redact, sign, and fill documents using the Nutrient DWS API. Works with PDFs, DOCX, XLSX, PPTX, HTML, and images.
---
# Nutrient Document Processing
Process documents with the [Nutrient DWS Processor API](https://www.nutrient.io/api/). Convert formats, extract text and tables, OCR scanned documents, redact PII, add watermarks, digitally sign, and fill PDF forms.
## Setup
Get a free API key at **[nutrient.io](https://dashboard.nutrient.io/sign_up/?product=processor)**
```bash
export NUTRIENT_API_KEY="pdf_live_..."
```
All requests go to `https://api.nutrient.io/build` as multipart POST with an `instructions` JSON field.
## Operations
### Convert Documents
```bash
# DOCX to PDF
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.docx=@document.docx" \
-F 'instructions={"parts":[{"file":"document.docx"}]}' \
-o output.pdf
# PDF to DOCX
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"output":{"type":"docx"}}' \
-o output.docx
# HTML to PDF
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "index.html=@index.html" \
-F 'instructions={"parts":[{"html":"index.html"}]}' \
-o output.pdf
```
Supported inputs: PDF, DOCX, XLSX, PPTX, DOC, XLS, PPT, PPS, PPSX, ODT, RTF, HTML, JPG, PNG, TIFF, HEIC, GIF, WebP, SVG, TGA, EPS.
### Extract Text and Data
```bash
# Extract plain text
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"output":{"type":"text"}}' \
-o output.txt
# Extract tables as Excel
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"output":{"type":"xlsx"}}' \
-o tables.xlsx
```
### OCR Scanned Documents
```bash
# OCR to searchable PDF (supports 100+ languages)
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "scanned.pdf=@scanned.pdf" \
-F 'instructions={"parts":[{"file":"scanned.pdf"}],"actions":[{"type":"ocr","language":"english"}]}' \
-o searchable.pdf
```
Languages: Supports 100+ languages via ISO 639-2 codes (e.g., `eng`, `deu`, `fra`, `spa`, `jpn`, `kor`, `chi_sim`, `chi_tra`, `ara`, `hin`, `rus`). Full language names like `english` or `german` also work. See the [complete OCR language table](https://www.nutrient.io/guides/document-engine/ocr/language-support/) for all supported codes.
### Redact Sensitive Information
```bash
# Pattern-based (SSN, email)
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"redaction","strategy":"preset","strategyOptions":{"preset":"social-security-number"}},{"type":"redaction","strategy":"preset","strategyOptions":{"preset":"email-address"}}]}' \
-o redacted.pdf
# Regex-based
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"redaction","strategy":"regex","strategyOptions":{"regex":"\\b[A-Z]{2}\\d{6}\\b"}}]}' \
-o redacted.pdf
```
Presets: `social-security-number`, `email-address`, `credit-card-number`, `international-phone-number`, `north-american-phone-number`, `date`, `time`, `url`, `ipv4`, `ipv6`, `mac-address`, `us-zip-code`, `vin`.
### Add Watermarks
```bash
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"watermark","text":"CONFIDENTIAL","fontSize":72,"opacity":0.3,"rotation":-45}]}' \
-o watermarked.pdf
```
### Digital Signatures
```bash
# Self-signed CMS signature
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "document.pdf=@document.pdf" \
-F 'instructions={"parts":[{"file":"document.pdf"}],"actions":[{"type":"sign","signatureType":"cms"}]}' \
-o signed.pdf
```
### Fill PDF Forms
```bash
curl -X POST https://api.nutrient.io/build \
-H "Authorization: Bearer $NUTRIENT_API_KEY" \
-F "form.pdf=@form.pdf" \
-F 'instructions={"parts":[{"file":"form.pdf"}],"actions":[{"type":"fillForm","formFields":{"name":"Jane Smith","email":"jane@example.com","date":"2026-02-06"}}]}' \
-o filled.pdf
```
## MCP Server (Alternative)
For native tool integration, use the MCP server instead of curl:
```json
{
"mcpServers": {
"nutrient-dws": {
"command": "npx",
"args": ["-y", "@nutrient-sdk/dws-mcp-server"],
"env": {
"NUTRIENT_DWS_API_KEY": "YOUR_API_KEY",
"SANDBOX_PATH": "/path/to/working/directory"
}
}
}
}
```
## When to Use
- Converting documents between formats (PDF, DOCX, XLSX, PPTX, HTML, images)
- Extracting text, tables, or key-value pairs from PDFs
- OCR on scanned documents or images
- Redacting PII before sharing documents
- Adding watermarks to drafts or confidential documents
- Digitally signing contracts or agreements
- Filling PDF forms programmatically
## Links
- [API Playground](https://dashboard.nutrient.io/processor-api/playground/)
- [Full API Docs](https://www.nutrient.io/guides/dws-processor/)
- [Agent Skill Repo](https://github.com/PSPDFKit-labs/nutrient-agent-skill)
- [npm MCP Server](https://www.npmjs.com/package/@nutrient-sdk/dws-mcp-server)

146
skills/postgres-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,146 @@
---
name: postgres-patterns
description: PostgreSQL database patterns for query optimization, schema design, indexing, and security. Based on Supabase best practices.
---
# PostgreSQL Patterns
Quick reference for PostgreSQL best practices. For detailed guidance, use the `database-reviewer` agent.
## When to Activate
- Writing SQL queries or migrations
- Designing database schemas
- Troubleshooting slow queries
- Implementing Row Level Security
- Setting up connection pooling
## Quick Reference
### Index Cheat Sheet
| Query Pattern | Index Type | Example |
|--------------|------------|---------|
| `WHERE col = value` | B-tree (default) | `CREATE INDEX idx ON t (col)` |
| `WHERE col > value` | B-tree | `CREATE INDEX idx ON t (col)` |
| `WHERE a = x AND b > y` | Composite | `CREATE INDEX idx ON t (a, b)` |
| `WHERE jsonb @> '{}'` | GIN | `CREATE INDEX idx ON t USING gin (col)` |
| `WHERE tsv @@ query` | GIN | `CREATE INDEX idx ON t USING gin (col)` |
| Time-series ranges | BRIN | `CREATE INDEX idx ON t USING brin (col)` |
### Data Type Quick Reference
| Use Case | Correct Type | Avoid |
|----------|-------------|-------|
| IDs | `bigint` | `int`, random UUID |
| Strings | `text` | `varchar(255)` |
| Timestamps | `timestamptz` | `timestamp` |
| Money | `numeric(10,2)` | `float` |
| Flags | `boolean` | `varchar`, `int` |
### Common Patterns
**Composite Index Order:**
```sql
-- Equality columns first, then range columns
CREATE INDEX idx ON orders (status, created_at);
-- Works for: WHERE status = 'pending' AND created_at > '2024-01-01'
```
**Covering Index:**
```sql
CREATE INDEX idx ON users (email) INCLUDE (name, created_at);
-- Avoids table lookup for SELECT email, name, created_at
```
**Partial Index:**
```sql
CREATE INDEX idx ON users (email) WHERE deleted_at IS NULL;
-- Smaller index, only includes active users
```
**RLS Policy (Optimized):**
```sql
CREATE POLICY policy ON orders
USING ((SELECT auth.uid()) = user_id); -- Wrap in SELECT!
```
**UPSERT:**
```sql
INSERT INTO settings (user_id, key, value)
VALUES (123, 'theme', 'dark')
ON CONFLICT (user_id, key)
DO UPDATE SET value = EXCLUDED.value;
```
**Cursor Pagination:**
```sql
SELECT * FROM products WHERE id > $last_id ORDER BY id LIMIT 20;
-- O(1) vs OFFSET which is O(n)
```
**Queue Processing:**
```sql
UPDATE jobs SET status = 'processing'
WHERE id = (
SELECT id FROM jobs WHERE status = 'pending'
ORDER BY created_at LIMIT 1
FOR UPDATE SKIP LOCKED
) RETURNING *;
```
### Anti-Pattern Detection
```sql
-- Find unindexed foreign keys
SELECT conrelid::regclass, a.attname
FROM pg_constraint c
JOIN pg_attribute a ON a.attrelid = c.conrelid AND a.attnum = ANY(c.conkey)
WHERE c.contype = 'f'
AND NOT EXISTS (
SELECT 1 FROM pg_index i
WHERE i.indrelid = c.conrelid AND a.attnum = ANY(i.indkey)
);
-- Find slow queries
SELECT query, mean_exec_time, calls
FROM pg_stat_statements
WHERE mean_exec_time > 100
ORDER BY mean_exec_time DESC;
-- Check table bloat
SELECT relname, n_dead_tup, last_vacuum
FROM pg_stat_user_tables
WHERE n_dead_tup > 1000
ORDER BY n_dead_tup DESC;
```
### Configuration Template
```sql
-- Connection limits (adjust for RAM)
ALTER SYSTEM SET max_connections = 100;
ALTER SYSTEM SET work_mem = '8MB';
-- Timeouts
ALTER SYSTEM SET idle_in_transaction_session_timeout = '30s';
ALTER SYSTEM SET statement_timeout = '30s';
-- Monitoring
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- Security defaults
REVOKE ALL ON SCHEMA public FROM public;
SELECT pg_reload_conf();
```
## Related
- Agent: `database-reviewer` - Full database review workflow
- Skill: `clickhouse-io` - ClickHouse analytics patterns
- Skill: `backend-patterns` - API and backend patterns
---
*Based on [Supabase Agent Skills](https://github.com/supabase/agent-skills) (MIT License)*

348
skills/project-guidelines-example/SKILL.md Normal file
View File

@@ -0,0 +1,348 @@
---
name: project-guidelines-example
description: "Example project-specific skill template based on a real production application."
---
# Project Guidelines Skill (Example)
This is an example of a project-specific skill. Use this as a template for your own projects.
Based on a real production application: [Zenith](https://zenith.chat) - AI-powered customer discovery platform.
## When to Use
Reference this skill when working on the specific project it's designed for. Project skills contain:
- Architecture overview
- File structure
- Code patterns
- Testing requirements
- Deployment workflow
---
## Architecture Overview
**Tech Stack:**
- **Frontend**: Next.js 15 (App Router), TypeScript, React
- **Backend**: FastAPI (Python), Pydantic models
- **Database**: Supabase (PostgreSQL)
- **AI**: Claude API with tool calling and structured output
- **Deployment**: Google Cloud Run
- **Testing**: Playwright (E2E), pytest (backend), React Testing Library
**Services:**
```
┌─────────────────────────────────────────────────────────────┐
│ Frontend │
│ Next.js 15 + TypeScript + TailwindCSS │
│ Deployed: Vercel / Cloud Run │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Backend │
│ FastAPI + Python 3.11 + Pydantic │
│ Deployed: Cloud Run │
└─────────────────────────────────────────────────────────────┘
│
┌───────────────┼───────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Supabase │ │ Claude │ │ Redis │
│ Database │ │ API │ │ Cache │
└──────────┘ └──────────┘ └──────────┘
```
---
## File Structure
```
project/
├── frontend/
│ └── src/
│ ├── app/ # Next.js app router pages
│ │ ├── api/ # API routes
│ │ ├── (auth)/ # Auth-protected routes
│ │ └── workspace/ # Main app workspace
│ ├── components/ # React components
│ │ ├── ui/ # Base UI components
│ │ ├── forms/ # Form components
│ │ └── layouts/ # Layout components
│ ├── hooks/ # Custom React hooks
│ ├── lib/ # Utilities
│ ├── types/ # TypeScript definitions
│ └── config/ # Configuration
│
├── backend/
│ ├── routers/ # FastAPI route handlers
│ ├── models.py # Pydantic models
│ ├── main.py # FastAPI app entry
│ ├── auth_system.py # Authentication
│ ├── database.py # Database operations
│ ├── services/ # Business logic
│ └── tests/ # pytest tests
│
├── deploy/ # Deployment configs
├── docs/ # Documentation
└── scripts/ # Utility scripts
```
---
## Code Patterns
### API Response Format (FastAPI)
```python
from pydantic import BaseModel
from typing import Generic, TypeVar, Optional
T = TypeVar('T')
class ApiResponse(BaseModel, Generic[T]):
success: bool
data: Optional[T] = None
error: Optional[str] = None
@classmethod
def ok(cls, data: T) -> "ApiResponse[T]":
return cls(success=True, data=data)
@classmethod
def fail(cls, error: str) -> "ApiResponse[T]":
return cls(success=False, error=error)
```
### Frontend API Calls (TypeScript)
```typescript
interface ApiResponse<T> {
success: boolean
data?: T
error?: string
}
async function fetchApi<T>(
endpoint: string,
options?: RequestInit
): Promise<ApiResponse<T>> {
try {
const response = await fetch(`/api${endpoint}`, {
...options,
headers: {
'Content-Type': 'application/json',
...options?.headers,
},
})
if (!response.ok) {
return { success: false, error: `HTTP ${response.status}` }
}
return await response.json()
} catch (error) {
return { success: false, error: String(error) }
}
}
```
### Claude AI Integration (Structured Output)
```python
from anthropic import Anthropic
from pydantic import BaseModel
class AnalysisResult(BaseModel):
summary: str
key_points: list[str]
confidence: float
async def analyze_with_claude(content: str) -> AnalysisResult:
client = Anthropic()
response = client.messages.create(
model="claude-sonnet-4-5-20250514",
max_tokens=1024,
messages=[{"role": "user", "content": content}],
tools=[{
"name": "provide_analysis",
"description": "Provide structured analysis",
"input_schema": AnalysisResult.model_json_schema()
}],
tool_choice={"type": "tool", "name": "provide_analysis"}
)
# Extract tool use result
tool_use = next(
block for block in response.content
if block.type == "tool_use"
)
return AnalysisResult(**tool_use.input)
```
### Custom Hooks (React)
```typescript
import { useState, useCallback } from 'react'
interface UseApiState<T> {
data: T | null
loading: boolean
error: string | null
}
export function useApi<T>(
fetchFn: () => Promise<ApiResponse<T>>
) {
const [state, setState] = useState<UseApiState<T>>({
data: null,
loading: false,
error: null,
})
const execute = useCallback(async () => {
setState(prev => ({ ...prev, loading: true, error: null }))
const result = await fetchFn()
if (result.success) {
setState({ data: result.data!, loading: false, error: null })
} else {
setState({ data: null, loading: false, error: result.error! })
}
}, [fetchFn])
return { ...state, execute }
}
```
---
## Testing Requirements
### Backend (pytest)
```bash
# Run all tests
poetry run pytest tests/
# Run with coverage
poetry run pytest tests/ --cov=. --cov-report=html
# Run specific test file
poetry run pytest tests/test_auth.py -v
```
**Test structure:**
```python
import pytest
from httpx import AsyncClient
from main import app
@pytest.fixture
async def client():
async with AsyncClient(app=app, base_url="http://test") as ac:
yield ac
@pytest.mark.asyncio
async def test_health_check(client: AsyncClient):
response = await client.get("/health")
assert response.status_code == 200
assert response.json()["status"] == "healthy"
```
### Frontend (React Testing Library)
```bash
# Run tests
npm run test
# Run with coverage
npm run test -- --coverage
# Run E2E tests
npm run test:e2e
```
**Test structure:**
```typescript
import { render, screen, fireEvent } from '@testing-library/react'
import { WorkspacePanel } from './WorkspacePanel'
describe('WorkspacePanel', () => {
it('renders workspace correctly', () => {
render(<WorkspacePanel />)
expect(screen.getByRole('main')).toBeInTheDocument()
})
it('handles session creation', async () => {
render(<WorkspacePanel />)
fireEvent.click(screen.getByText('New Session'))
expect(await screen.findByText('Session created')).toBeInTheDocument()
})
})
```
---
## Deployment Workflow
### Pre-Deployment Checklist
- [ ] All tests passing locally
- [ ] `npm run build` succeeds (frontend)
- [ ] `poetry run pytest` passes (backend)
- [ ] No hardcoded secrets
- [ ] Environment variables documented
- [ ] Database migrations ready
### Deployment Commands
```bash
# Build and deploy frontend
cd frontend && npm run build
gcloud run deploy frontend --source .
# Build and deploy backend
cd backend
gcloud run deploy backend --source .
```
### Environment Variables
```bash
# Frontend (.env.local)
NEXT_PUBLIC_API_URL=https://api.example.com
NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
# Backend (.env)
DATABASE_URL=postgresql://...
ANTHROPIC_API_KEY=sk-ant-...
SUPABASE_URL=https://xxx.supabase.co
SUPABASE_KEY=eyJ...
```
---
## Critical Rules
1. **No emojis** in code, comments, or documentation
2. **Immutability** - never mutate objects or arrays
3. **TDD** - write tests before implementation
4. **80% coverage** minimum
5. **Many small files** - 200-400 lines typical, 800 max
6. **No console.log** in production code
7. **Proper error handling** with try/catch
8. **Input validation** with Pydantic/Zod
---
## Related Skills
- `coding-standards.md` - General coding best practices
- `backend-patterns.md` - API and database patterns
- `frontend-patterns.md` - React and Next.js patterns
- `tdd-workflow/` - Test-driven development methodology

749
skills/python-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,749 @@
---
name: python-patterns
description: Pythonic idioms, PEP 8 standards, type hints, and best practices for building robust, efficient, and maintainable Python applications.
---
# Python Development Patterns
Idiomatic Python patterns and best practices for building robust, efficient, and maintainable applications.
## When to Activate
- Writing new Python code
- Reviewing Python code
- Refactoring existing Python code
- Designing Python packages/modules
## Core Principles
### 1. Readability Counts
Python prioritizes readability. Code should be obvious and easy to understand.
```python
# Good: Clear and readable
def get_active_users(users: list[User]) -> list[User]:
"""Return only active users from the provided list."""
return [user for user in users if user.is_active]
# Bad: Clever but confusing
def get_active_users(u):
return [x for x in u if x.a]
```
### 2. Explicit is Better Than Implicit
Avoid magic; be clear about what your code does.
```python
# Good: Explicit configuration
import logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
# Bad: Hidden side effects
import some_module
some_module.setup() # What does this do?
```
### 3. EAFP - Easier to Ask Forgiveness Than Permission
Python prefers exception handling over checking conditions.
```python
# Good: EAFP style
def get_value(dictionary: dict, key: str) -> Any:
try:
return dictionary[key]
except KeyError:
return default_value
# Bad: LBYL (Look Before You Leap) style
def get_value(dictionary: dict, key: str) -> Any:
if key in dictionary:
return dictionary[key]
else:
return default_value
```
## Type Hints
### Basic Type Annotations
```python
from typing import Optional, List, Dict, Any
def process_user(
user_id: str,
data: Dict[str, Any],
active: bool = True
) -> Optional[User]:
"""Process a user and return the updated User or None."""
if not active:
return None
return User(user_id, data)
```
### Modern Type Hints (Python 3.9+)
```python
# Python 3.9+ - Use built-in types
def process_items(items: list[str]) -> dict[str, int]:
return {item: len(item) for item in items}
# Python 3.8 and earlier - Use typing module
from typing import List, Dict
def process_items(items: List[str]) -> Dict[str, int]:
return {item: len(item) for item in items}
```
### Type Aliases and TypeVar
```python
from typing import TypeVar, Union
# Type alias for complex types
JSON = Union[dict[str, Any], list[Any], str, int, float, bool, None]
def parse_json(data: str) -> JSON:
return json.loads(data)
# Generic types
T = TypeVar('T')
def first(items: list[T]) -> T | None:
"""Return the first item or None if list is empty."""
return items[0] if items else None
```
### Protocol-Based Duck Typing
```python
from typing import Protocol
class Renderable(Protocol):
def render(self) -> str:
"""Render the object to a string."""
def render_all(items: list[Renderable]) -> str:
"""Render all items that implement the Renderable protocol."""
return "\n".join(item.render() for item in items)
```
## Error Handling Patterns
### Specific Exception Handling
```python
# Good: Catch specific exceptions
def load_config(path: str) -> Config:
try:
with open(path) as f:
return Config.from_json(f.read())
except FileNotFoundError as e:
raise ConfigError(f"Config file not found: {path}") from e
except json.JSONDecodeError as e:
raise ConfigError(f"Invalid JSON in config: {path}") from e
# Bad: Bare except
def load_config(path: str) -> Config:
try:
with open(path) as f:
return Config.from_json(f.read())
except:
return None # Silent failure!
```
### Exception Chaining
```python
def process_data(data: str) -> Result:
try:
parsed = json.loads(data)
except json.JSONDecodeError as e:
# Chain exceptions to preserve the traceback
raise ValueError(f"Failed to parse data: {data}") from e
```
### Custom Exception Hierarchy
```python
class AppError(Exception):
"""Base exception for all application errors."""
pass
class ValidationError(AppError):
"""Raised when input validation fails."""
pass
class NotFoundError(AppError):
"""Raised when a requested resource is not found."""
pass
# Usage
def get_user(user_id: str) -> User:
user = db.find_user(user_id)
if not user:
raise NotFoundError(f"User not found: {user_id}")
return user
```
## Context Managers
### Resource Management
```python
# Good: Using context managers
def process_file(path: str) -> str:
with open(path, 'r') as f:
return f.read()
# Bad: Manual resource management
def process_file(path: str) -> str:
f = open(path, 'r')
try:
return f.read()
finally:
f.close()
```
### Custom Context Managers
```python
from contextlib import contextmanager
@contextmanager
def timer(name: str):
"""Context manager to time a block of code."""
start = time.perf_counter()
yield
elapsed = time.perf_counter() - start
print(f"{name} took {elapsed:.4f} seconds")
# Usage
with timer("data processing"):
process_large_dataset()
```
### Context Manager Classes
```python
class DatabaseTransaction:
def __init__(self, connection):
self.connection = connection
def __enter__(self):
self.connection.begin_transaction()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is None:
self.connection.commit()
else:
self.connection.rollback()
return False # Don't suppress exceptions
# Usage
with DatabaseTransaction(conn):
user = conn.create_user(user_data)
conn.create_profile(user.id, profile_data)
```
## Comprehensions and Generators
### List Comprehensions
```python
# Good: List comprehension for simple transformations
names = [user.name for user in users if user.is_active]
# Bad: Manual loop
names = []
for user in users:
if user.is_active:
names.append(user.name)
# Complex comprehensions should be expanded
# Bad: Too complex
result = [x * 2 for x in items if x > 0 if x % 2 == 0]
# Good: Use a generator function
def filter_and_transform(items: Iterable[int]) -> list[int]:
result = []
for x in items:
if x > 0 and x % 2 == 0:
result.append(x * 2)
return result
```
### Generator Expressions
```python
# Good: Generator for lazy evaluation
total = sum(x * x for x in range(1_000_000))
# Bad: Creates large intermediate list
total = sum([x * x for x in range(1_000_000)])
```
### Generator Functions
```python
def read_large_file(path: str) -> Iterator[str]:
"""Read a large file line by line."""
with open(path) as f:
for line in f:
yield line.strip()
# Usage
for line in read_large_file("huge.txt"):
process(line)
```
## Data Classes and Named Tuples
### Data Classes
```python
from dataclasses import dataclass, field
from datetime import datetime
@dataclass
class User:
"""User entity with automatic __init__, __repr__, and __eq__."""
id: str
name: str
email: str
created_at: datetime = field(default_factory=datetime.now)
is_active: bool = True
# Usage
user = User(
id="123",
name="Alice",
email="alice@example.com"
)
```
### Data Classes with Validation
```python
@dataclass
class User:
email: str
age: int
def __post_init__(self):
# Validate email format
if "@" not in self.email:
raise ValueError(f"Invalid email: {self.email}")
# Validate age range
if self.age < 0 or self.age > 150:
raise ValueError(f"Invalid age: {self.age}")
```
### Named Tuples
```python
from typing import NamedTuple
class Point(NamedTuple):
"""Immutable 2D point."""
x: float
y: float
def distance(self, other: 'Point') -> float:
return ((self.x - other.x) ** 2 + (self.y - other.y) ** 2) ** 0.5
# Usage
p1 = Point(0, 0)
p2 = Point(3, 4)
print(p1.distance(p2)) # 5.0
```
## Decorators
### Function Decorators
```python
import functools
import time
def timer(func: Callable) -> Callable:
"""Decorator to time function execution."""
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
elapsed = time.perf_counter() - start
print(f"{func.__name__} took {elapsed:.4f}s")
return result
return wrapper
@timer
def slow_function():
time.sleep(1)
# slow_function() prints: slow_function took 1.0012s
```
### Parameterized Decorators
```python
def repeat(times: int):
"""Decorator to repeat a function multiple times."""
def decorator(func: Callable) -> Callable:
@functools.wraps(func)
def wrapper(*args, **kwargs):
results = []
for _ in range(times):
results.append(func(*args, **kwargs))
return results
return wrapper
return decorator
@repeat(times=3)
def greet(name: str) -> str:
return f"Hello, {name}!"
# greet("Alice") returns ["Hello, Alice!", "Hello, Alice!", "Hello, Alice!"]
```
### Class-Based Decorators
```python
class CountCalls:
"""Decorator that counts how many times a function is called."""
def __init__(self, func: Callable):
functools.update_wrapper(self, func)
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"{self.func.__name__} has been called {self.count} times")
return self.func(*args, **kwargs)
@CountCalls
def process():
pass
# Each call to process() prints the call count
```
## Concurrency Patterns
### Threading for I/O-Bound Tasks
```python
import concurrent.futures
import threading
def fetch_url(url: str) -> str:
"""Fetch a URL (I/O-bound operation)."""
import urllib.request
with urllib.request.urlopen(url) as response:
return response.read().decode()
def fetch_all_urls(urls: list[str]) -> dict[str, str]:
"""Fetch multiple URLs concurrently using threads."""
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
future_to_url = {executor.submit(fetch_url, url): url for url in urls}
results = {}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
results[url] = future.result()
except Exception as e:
results[url] = f"Error: {e}"
return results
```
### Multiprocessing for CPU-Bound Tasks
```python
def process_data(data: list[int]) -> int:
"""CPU-intensive computation."""
return sum(x ** 2 for x in data)
def process_all(datasets: list[list[int]]) -> list[int]:
"""Process multiple datasets using multiple processes."""
with concurrent.futures.ProcessPoolExecutor() as executor:
results = list(executor.map(process_data, datasets))
return results
```
### Async/Await for Concurrent I/O
```python
import asyncio
async def fetch_async(url: str) -> str:
"""Fetch a URL asynchronously."""
import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.text()
async def fetch_all(urls: list[str]) -> dict[str, str]:
"""Fetch multiple URLs concurrently."""
tasks = [fetch_async(url) for url in urls]
results = await asyncio.gather(*tasks, return_exceptions=True)
return dict(zip(urls, results))
```
## Package Organization
### Standard Project Layout
```
myproject/
├── src/
│ └── mypackage/
│ ├── __init__.py
│ ├── main.py
│ ├── api/
│ │ ├── __init__.py
│ │ └── routes.py
│ ├── models/
│ │ ├── __init__.py
│ │ └── user.py
│ └── utils/
│ ├── __init__.py
│ └── helpers.py
├── tests/
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_api.py
│ └── test_models.py
├── pyproject.toml
├── README.md
└── .gitignore
```
### Import Conventions
```python
# Good: Import order - stdlib, third-party, local
import os
import sys
from pathlib import Path
import requests
from fastapi import FastAPI
from mypackage.models import User
from mypackage.utils import format_name
# Good: Use isort for automatic import sorting
# pip install isort
```
### __init__.py for Package Exports
```python
# mypackage/__init__.py
"""mypackage - A sample Python package."""
__version__ = "1.0.0"
# Export main classes/functions at package level
from mypackage.models import User, Post
from mypackage.utils import format_name
__all__ = ["User", "Post", "format_name"]
```
## Memory and Performance
### Using __slots__ for Memory Efficiency
```python
# Bad: Regular class uses __dict__ (more memory)
class Point:
def __init__(self, x: float, y: float):
self.x = x
self.y = y
# Good: __slots__ reduces memory usage
class Point:
__slots__ = ['x', 'y']
def __init__(self, x: float, y: float):
self.x = x
self.y = y
```
### Generator for Large Data
```python
# Bad: Returns full list in memory
def read_lines(path: str) -> list[str]:
with open(path) as f:
return [line.strip() for line in f]
# Good: Yields lines one at a time
def read_lines(path: str) -> Iterator[str]:
with open(path) as f:
for line in f:
yield line.strip()
```
### Avoid String Concatenation in Loops
```python
# Bad: O(n²) due to string immutability
result = ""
for item in items:
result += str(item)
# Good: O(n) using join
result = "".join(str(item) for item in items)
# Good: Using StringIO for building
from io import StringIO
buffer = StringIO()
for item in items:
buffer.write(str(item))
result = buffer.getvalue()
```
## Python Tooling Integration
### Essential Commands
```bash
# Code formatting
black .
isort .
# Linting
ruff check .
pylint mypackage/
# Type checking
mypy .
# Testing
pytest --cov=mypackage --cov-report=html
# Security scanning
bandit -r .
# Dependency management
pip-audit
safety check
```
### pyproject.toml Configuration
```toml
[project]
name = "mypackage"
version = "1.0.0"
requires-python = ">=3.9"
dependencies = [
"requests>=2.31.0",
"pydantic>=2.0.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.4.0",
"pytest-cov>=4.1.0",
"black>=23.0.0",
"ruff>=0.1.0",
"mypy>=1.5.0",
]
[tool.black]
line-length = 88
target-version = ['py39']
[tool.ruff]
line-length = 88
select = ["E", "F", "I", "N", "W"]
[tool.mypy]
python_version = "3.9"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--cov=mypackage --cov-report=term-missing"
```
## Quick Reference: Python Idioms
| Idiom | Description |
|-------|-------------|
| EAFP | Easier to Ask Forgiveness than Permission |
| Context managers | Use `with` for resource management |
| List comprehensions | For simple transformations |
| Generators | For lazy evaluation and large datasets |
| Type hints | Annotate function signatures |
| Dataclasses | For data containers with auto-generated methods |
| `__slots__` | For memory optimization |
| f-strings | For string formatting (Python 3.6+) |
| `pathlib.Path` | For path operations (Python 3.4+) |
| `enumerate` | For index-element pairs in loops |
## Anti-Patterns to Avoid
```python
# Bad: Mutable default arguments
def append_to(item, items=[]):
items.append(item)
return items
# Good: Use None and create new list
def append_to(item, items=None):
if items is None:
items = []
items.append(item)
return items
# Bad: Checking type with type()
if type(obj) == list:
process(obj)
# Good: Use isinstance
if isinstance(obj, list):
process(obj)
# Bad: Comparing to None with ==
if value == None:
process()
# Good: Use is
if value is None:
process()
# Bad: from module import *
from os.path import *
# Good: Explicit imports
from os.path import join, exists
# Bad: Bare except
try:
risky_operation()
except:
pass
# Good: Specific exception
try:
risky_operation()
except SpecificError as e:
logger.error(f"Operation failed: {e}")
```
__Remember__: Python code should be readable, explicit, and follow the principle of least surprise. When in doubt, prioritize clarity over cleverness.

815
skills/python-testing/SKILL.md Normal file
View File

@@ -0,0 +1,815 @@
---
name: python-testing
description: Python testing strategies using pytest, TDD methodology, fixtures, mocking, parametrization, and coverage requirements.
---
# Python Testing Patterns
Comprehensive testing strategies for Python applications using pytest, TDD methodology, and best practices.
## When to Activate
- Writing new Python code (follow TDD: red, green, refactor)
- Designing test suites for Python projects
- Reviewing Python test coverage
- Setting up testing infrastructure
## Core Testing Philosophy
### Test-Driven Development (TDD)
Always follow the TDD cycle:
1. **RED**: Write a failing test for the desired behavior
2. **GREEN**: Write minimal code to make the test pass
3. **REFACTOR**: Improve code while keeping tests green
```python
# Step 1: Write failing test (RED)
def test_add_numbers():
result = add(2, 3)
assert result == 5
# Step 2: Write minimal implementation (GREEN)
def add(a, b):
return a + b
# Step 3: Refactor if needed (REFACTOR)
```
### Coverage Requirements
- **Target**: 80%+ code coverage
- **Critical paths**: 100% coverage required
- Use `pytest --cov` to measure coverage
```bash
pytest --cov=mypackage --cov-report=term-missing --cov-report=html
```
## pytest Fundamentals
### Basic Test Structure
```python
import pytest
def test_addition():
"""Test basic addition."""
assert 2 + 2 == 4
def test_string_uppercase():
"""Test string uppercasing."""
text = "hello"
assert text.upper() == "HELLO"
def test_list_append():
"""Test list append."""
items = [1, 2, 3]
items.append(4)
assert 4 in items
assert len(items) == 4
```
### Assertions
```python
# Equality
assert result == expected
# Inequality
assert result != unexpected
# Truthiness
assert result # Truthy
assert not result # Falsy
assert result is True # Exactly True
assert result is False # Exactly False
assert result is None # Exactly None
# Membership
assert item in collection
assert item not in collection
# Comparisons
assert result > 0
assert 0 <= result <= 100
# Type checking
assert isinstance(result, str)
# Exception testing (preferred approach)
with pytest.raises(ValueError):
raise ValueError("error message")
# Check exception message
with pytest.raises(ValueError, match="invalid input"):
raise ValueError("invalid input provided")
# Check exception attributes
with pytest.raises(ValueError) as exc_info:
raise ValueError("error message")
assert str(exc_info.value) == "error message"
```
## Fixtures
### Basic Fixture Usage
```python
import pytest
@pytest.fixture
def sample_data():
"""Fixture providing sample data."""
return {"name": "Alice", "age": 30}
def test_sample_data(sample_data):
"""Test using the fixture."""
assert sample_data["name"] == "Alice"
assert sample_data["age"] == 30
```
### Fixture with Setup/Teardown
```python
@pytest.fixture
def database():
"""Fixture with setup and teardown."""
# Setup
db = Database(":memory:")
db.create_tables()
db.insert_test_data()
yield db # Provide to test
# Teardown
db.close()
def test_database_query(database):
"""Test database operations."""
result = database.query("SELECT * FROM users")
assert len(result) > 0
```
### Fixture Scopes
```python
# Function scope (default) - runs for each test
@pytest.fixture
def temp_file():
with open("temp.txt", "w") as f:
yield f
os.remove("temp.txt")
# Module scope - runs once per module
@pytest.fixture(scope="module")
def module_db():
db = Database(":memory:")
db.create_tables()
yield db
db.close()
# Session scope - runs once per test session
@pytest.fixture(scope="session")
def shared_resource():
resource = ExpensiveResource()
yield resource
resource.cleanup()
```
### Fixture with Parameters
```python
@pytest.fixture(params=[1, 2, 3])
def number(request):
"""Parameterized fixture."""
return request.param
def test_numbers(number):
"""Test runs 3 times, once for each parameter."""
assert number > 0
```
### Using Multiple Fixtures
```python
@pytest.fixture
def user():
return User(id=1, name="Alice")
@pytest.fixture
def admin():
return User(id=2, name="Admin", role="admin")
def test_user_admin_interaction(user, admin):
"""Test using multiple fixtures."""
assert admin.can_manage(user)
```
### Autouse Fixtures
```python
@pytest.fixture(autouse=True)
def reset_config():
"""Automatically runs before every test."""
Config.reset()
yield
Config.cleanup()
def test_without_fixture_call():
# reset_config runs automatically
assert Config.get_setting("debug") is False
```
### Conftest.py for Shared Fixtures
```python
# tests/conftest.py
import pytest
@pytest.fixture
def client():
"""Shared fixture for all tests."""
app = create_app(testing=True)
with app.test_client() as client:
yield client
@pytest.fixture
def auth_headers(client):
"""Generate auth headers for API testing."""
response = client.post("/api/login", json={
"username": "test",
"password": "test"
})
token = response.json["token"]
return {"Authorization": f"Bearer {token}"}
```
## Parametrization
### Basic Parametrization
```python
@pytest.mark.parametrize("input,expected", [
("hello", "HELLO"),
("world", "WORLD"),
("PyThOn", "PYTHON"),
])
def test_uppercase(input, expected):
"""Test runs 3 times with different inputs."""
assert input.upper() == expected
```
### Multiple Parameters
```python
@pytest.mark.parametrize("a,b,expected", [
(2, 3, 5),
(0, 0, 0),
(-1, 1, 0),
(100, 200, 300),
])
def test_add(a, b, expected):
"""Test addition with multiple inputs."""
assert add(a, b) == expected
```
### Parametrize with IDs
```python
@pytest.mark.parametrize("input,expected", [
("valid@email.com", True),
("invalid", False),
("@no-domain.com", False),
], ids=["valid-email", "missing-at", "missing-domain"])
def test_email_validation(input, expected):
"""Test email validation with readable test IDs."""
assert is_valid_email(input) is expected
```
### Parametrized Fixtures
```python
@pytest.fixture(params=["sqlite", "postgresql", "mysql"])
def db(request):
"""Test against multiple database backends."""
if request.param == "sqlite":
return Database(":memory:")
elif request.param == "postgresql":
return Database("postgresql://localhost/test")
elif request.param == "mysql":
return Database("mysql://localhost/test")
def test_database_operations(db):
"""Test runs 3 times, once for each database."""
result = db.query("SELECT 1")
assert result is not None
```
## Markers and Test Selection
### Custom Markers
```python
# Mark slow tests
@pytest.mark.slow
def test_slow_operation():
time.sleep(5)
# Mark integration tests
@pytest.mark.integration
def test_api_integration():
response = requests.get("https://api.example.com")
assert response.status_code == 200
# Mark unit tests
@pytest.mark.unit
def test_unit_logic():
assert calculate(2, 3) == 5
```
### Run Specific Tests
```bash
# Run only fast tests
pytest -m "not slow"
# Run only integration tests
pytest -m integration
# Run integration or slow tests
pytest -m "integration or slow"
# Run tests marked as unit but not slow
pytest -m "unit and not slow"
```
### Configure Markers in pytest.ini
```ini
[pytest]
markers =
slow: marks tests as slow
integration: marks tests as integration tests
unit: marks tests as unit tests
django: marks tests as requiring Django
```
## Mocking and Patching
### Mocking Functions
```python
from unittest.mock import patch, Mock
@patch("mypackage.external_api_call")
def test_with_mock(api_call_mock):
"""Test with mocked external API."""
api_call_mock.return_value = {"status": "success"}
result = my_function()
api_call_mock.assert_called_once()
assert result["status"] == "success"
```
### Mocking Return Values
```python
@patch("mypackage.Database.connect")
def test_database_connection(connect_mock):
"""Test with mocked database connection."""
connect_mock.return_value = MockConnection()
db = Database()
db.connect()
connect_mock.assert_called_once_with("localhost")
```
### Mocking Exceptions
```python
@patch("mypackage.api_call")
def test_api_error_handling(api_call_mock):
"""Test error handling with mocked exception."""
api_call_mock.side_effect = ConnectionError("Network error")
with pytest.raises(ConnectionError):
api_call()
api_call_mock.assert_called_once()
```
### Mocking Context Managers
```python
@patch("builtins.open", new_callable=mock_open)
def test_file_reading(mock_file):
"""Test file reading with mocked open."""
mock_file.return_value.read.return_value = "file content"
result = read_file("test.txt")
mock_file.assert_called_once_with("test.txt", "r")
assert result == "file content"
```
### Using Autospec
```python
@patch("mypackage.DBConnection", autospec=True)
def test_autospec(db_mock):
"""Test with autospec to catch API misuse."""
db = db_mock.return_value
db.query("SELECT * FROM users")
# This would fail if DBConnection doesn't have query method
db_mock.assert_called_once()
```
### Mock Class Instances
```python
class TestUserService:
@patch("mypackage.UserRepository")
def test_create_user(self, repo_mock):
"""Test user creation with mocked repository."""
repo_mock.return_value.save.return_value = User(id=1, name="Alice")
service = UserService(repo_mock.return_value)
user = service.create_user(name="Alice")
assert user.name == "Alice"
repo_mock.return_value.save.assert_called_once()
```
### Mock Property
```python
@pytest.fixture
def mock_config():
"""Create a mock with a property."""
config = Mock()
type(config).debug = PropertyMock(return_value=True)
type(config).api_key = PropertyMock(return_value="test-key")
return config
def test_with_mock_config(mock_config):
"""Test with mocked config properties."""
assert mock_config.debug is True
assert mock_config.api_key == "test-key"
```
## Testing Async Code
### Async Tests with pytest-asyncio
```python
import pytest
@pytest.mark.asyncio
async def test_async_function():
"""Test async function."""
result = await async_add(2, 3)
assert result == 5
@pytest.mark.asyncio
async def test_async_with_fixture(async_client):
"""Test async with async fixture."""
response = await async_client.get("/api/users")
assert response.status_code == 200
```
### Async Fixture
```python
@pytest.fixture
async def async_client():
"""Async fixture providing async test client."""
app = create_app()
async with app.test_client() as client:
yield client
@pytest.mark.asyncio
async def test_api_endpoint(async_client):
"""Test using async fixture."""
response = await async_client.get("/api/data")
assert response.status_code == 200
```
### Mocking Async Functions
```python
@pytest.mark.asyncio
@patch("mypackage.async_api_call")
async def test_async_mock(api_call_mock):
"""Test async function with mock."""
api_call_mock.return_value = {"status": "ok"}
result = await my_async_function()
api_call_mock.assert_awaited_once()
assert result["status"] == "ok"
```
## Testing Exceptions
### Testing Expected Exceptions
```python
def test_divide_by_zero():
"""Test that dividing by zero raises ZeroDivisionError."""
with pytest.raises(ZeroDivisionError):
divide(10, 0)
def test_custom_exception():
"""Test custom exception with message."""
with pytest.raises(ValueError, match="invalid input"):
validate_input("invalid")
```
### Testing Exception Attributes
```python
def test_exception_with_details():
"""Test exception with custom attributes."""
with pytest.raises(CustomError) as exc_info:
raise CustomError("error", code=400)
assert exc_info.value.code == 400
assert "error" in str(exc_info.value)
```
## Testing Side Effects
### Testing File Operations
```python
import tempfile
import os
def test_file_processing():
"""Test file processing with temp file."""
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt') as f:
f.write("test content")
temp_path = f.name
try:
result = process_file(temp_path)
assert result == "processed: test content"
finally:
os.unlink(temp_path)
```
### Testing with pytest's tmp_path Fixture
```python
def test_with_tmp_path(tmp_path):
"""Test using pytest's built-in temp path fixture."""
test_file = tmp_path / "test.txt"
test_file.write_text("hello world")
result = process_file(str(test_file))
assert result == "hello world"
# tmp_path automatically cleaned up
```
### Testing with tmpdir Fixture
```python
def test_with_tmpdir(tmpdir):
"""Test using pytest's tmpdir fixture."""
test_file = tmpdir.join("test.txt")
test_file.write("data")
result = process_file(str(test_file))
assert result == "data"
```
## Test Organization
### Directory Structure
```
tests/
├── conftest.py # Shared fixtures
├── __init__.py
├── unit/ # Unit tests
│ ├── __init__.py
│ ├── test_models.py
│ ├── test_utils.py
│ └── test_services.py
├── integration/ # Integration tests
│ ├── __init__.py
│ ├── test_api.py
│ └── test_database.py
└── e2e/ # End-to-end tests
├── __init__.py
└── test_user_flow.py
```
### Test Classes
```python
class TestUserService:
"""Group related tests in a class."""
@pytest.fixture(autouse=True)
def setup(self):
"""Setup runs before each test in this class."""
self.service = UserService()
def test_create_user(self):
"""Test user creation."""
user = self.service.create_user("Alice")
assert user.name == "Alice"
def test_delete_user(self):
"""Test user deletion."""
user = User(id=1, name="Bob")
self.service.delete_user(user)
assert not self.service.user_exists(1)
```
## Best Practices
### DO
- **Follow TDD**: Write tests before code (red-green-refactor)
- **Test one thing**: Each test should verify a single behavior
- **Use descriptive names**: `test_user_login_with_invalid_credentials_fails`
- **Use fixtures**: Eliminate duplication with fixtures
- **Mock external dependencies**: Don't depend on external services
- **Test edge cases**: Empty inputs, None values, boundary conditions
- **Aim for 80%+ coverage**: Focus on critical paths
- **Keep tests fast**: Use marks to separate slow tests
### DON'T
- **Don't test implementation**: Test behavior, not internals
- **Don't use complex conditionals in tests**: Keep tests simple
- **Don't ignore test failures**: All tests must pass
- **Don't test third-party code**: Trust libraries to work
- **Don't share state between tests**: Tests should be independent
- **Don't catch exceptions in tests**: Use `pytest.raises`
- **Don't use print statements**: Use assertions and pytest output
- **Don't write tests that are too brittle**: Avoid over-specific mocks
## Common Patterns
### Testing API Endpoints (FastAPI/Flask)
```python
@pytest.fixture
def client():
app = create_app(testing=True)
return app.test_client()
def test_get_user(client):
response = client.get("/api/users/1")
assert response.status_code == 200
assert response.json["id"] == 1
def test_create_user(client):
response = client.post("/api/users", json={
"name": "Alice",
"email": "alice@example.com"
})
assert response.status_code == 201
assert response.json["name"] == "Alice"
```
### Testing Database Operations
```python
@pytest.fixture
def db_session():
"""Create a test database session."""
session = Session(bind=engine)
session.begin_nested()
yield session
session.rollback()
session.close()
def test_create_user(db_session):
user = User(name="Alice", email="alice@example.com")
db_session.add(user)
db_session.commit()
retrieved = db_session.query(User).filter_by(name="Alice").first()
assert retrieved.email == "alice@example.com"
```
### Testing Class Methods
```python
class TestCalculator:
@pytest.fixture
def calculator(self):
return Calculator()
def test_add(self, calculator):
assert calculator.add(2, 3) == 5
def test_divide_by_zero(self, calculator):
with pytest.raises(ZeroDivisionError):
calculator.divide(10, 0)
```
## pytest Configuration
### pytest.ini
```ini
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
--strict-markers
--disable-warnings
--cov=mypackage
--cov-report=term-missing
--cov-report=html
markers =
slow: marks tests as slow
integration: marks tests as integration tests
unit: marks tests as unit tests
```
### pyproject.toml
```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"--strict-markers",
"--cov=mypackage",
"--cov-report=term-missing",
"--cov-report=html",
]
markers = [
"slow: marks tests as slow",
"integration: marks tests as integration tests",
"unit: marks tests as unit tests",
]
```
## Running Tests
```bash
# Run all tests
pytest
# Run specific file
pytest tests/test_utils.py
# Run specific test
pytest tests/test_utils.py::test_function
# Run with verbose output
pytest -v
# Run with coverage
pytest --cov=mypackage --cov-report=html
# Run only fast tests
pytest -m "not slow"
# Run until first failure
pytest -x
# Run and stop on N failures
pytest --maxfail=3
# Run last failed tests
pytest --lf
# Run tests with pattern
pytest -k "test_user"
# Run with debugger on failure
pytest --pdb
```
## Quick Reference
| Pattern | Usage |
|---------|-------|
| `pytest.raises()` | Test expected exceptions |
| `@pytest.fixture()` | Create reusable test fixtures |
| `@pytest.mark.parametrize()` | Run tests with multiple inputs |
| `@pytest.mark.slow` | Mark slow tests |
| `pytest -m "not slow"` | Skip slow tests |
| `@patch()` | Mock functions and classes |
| `tmp_path` fixture | Automatic temp directory |
| `pytest --cov` | Generate coverage report |
| `assert` | Simple and readable assertions |
**Remember**: Tests are code too. Keep them clean, readable, and maintainable. Good tests catch bugs; great tests prevent them.

44
skills/python.md Normal file
View File

@@ -0,0 +1,44 @@
# Python Development Skill
## Installed
- Python 3.12
- Python IDLE (idle-python3.12)
## Common Commands
### Virtual Environments
```bash
python3 -m venv venv # Create venv
source venv/bin/activate # Activate venv
pip install package # Install package
pip freeze > requirements.txt # Save deps
deactivate # Deactivate
```
### Common Tools
```bash
pip install -U pip # Upgrade pip
pip install requests # HTTP library
pip install flask # Web framework
pip install django # Full framework
pip install pytest # Testing
pip install black # Formatter
pip install ruff # Linter
```
### Run Python
```bash
python script.py # Run script
python -m http.server 8000 # HTTP server
python -c "print('hi')" # One-liner
```
### Docker for Python
```dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "app.py"]
```

56
skills/react-frontend.md Normal file
View File

@@ -0,0 +1,56 @@
# React & Frontend Development Skill
## Installed
- Node.js 20.x
- npm 10.x
## Quick Start
```bash
# Create React app
npx create-react-app myapp
cd myapp && npm start
# With Vite (faster)
npm create vite@latest myapp -- --template react
cd myapp && npm install && npm run dev
```
## Common Commands
```bash
npm run dev # Development server
npm run build # Production build
npm run preview # Preview production build
npm install package # Install dependency
```
## Key Libraries
```bash
# UI
npm install @mui/material @emotion/react @emotion/styled # Material UI
npm install tailwindcss postcss autoprefixer # Tailwind CSS
npm install shadcn-ui # shadcn/ui
# State
npm install zustand # State management
npm install @reduxjs/toolkit react-redux # Redux
# Routing
npm install react-router-dom
# HTTP
npm install axios
# Forms
npm install react-hook-form
```
## Docker for React
```dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
EXPOSE 3000
CMD ["npm", "start"]
```

58
skills/rocm-gpu.md Normal file
View File

@@ -0,0 +1,58 @@
# ROCm & AMD GPU Computing Skill
## Hardware
- GPU: AMD Radeon RX 6800 XT (Navi 21)
- VRAM: 16GB
- Architecture: gfx1030
- Kernel: 6.8.0-55-generic (with amdgpu-dkms)
## Installed ROCm Components
- ROCm 6.3.2
- rocminfo
- rocm-smi
- hip-runtime-amd
- miopen-hip
- rocblas
- rocfft
- rocrand
## Essential Commands
### GPU Monitoring
```bash
rocm-smi # GPU status
rocm-smi --showproductname # GPU info
rocm-smi --showpid # Show processes
rocminfo # Detailed ROCm info
```
### Environment Variables
```bash
# Add to ~/.bashrc for permanent setup
export PATH=/opt/rocm/bin:$PATH
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
```
### PyTorch with ROCm
```bash
# Install PyTorch for ROCm
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0
# Verify GPU access
python -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))"
```
### Docker with ROCm
```bash
docker run -it --device=/dev/kfd --device=/dev/dri --group-add video rocm/pytorch:latest
```
## Performance Tuning
- Set `HSA_OVERRIDE_GFX_VERSION=10.3.0` for compatibility
- GPU temperature: Check with `rocm-smi` (normal <85°C)
- Power limit: 264W (default)
## Common Issues
- "No ROCm-capable GPU": Check kernel is 6.8.0-55, not 6.17
- Missing libraries: Ensure /opt/rocm/lib is in LD_LIBRARY_PATH
- Permission denied: User must be in 'render' and 'video' groups

494
skills/security-review/SKILL.md Normal file
View File

@@ -0,0 +1,494 @@
---
name: security-review
description: Use this skill when adding authentication, handling user input, working with secrets, creating API endpoints, or implementing payment/sensitive features. Provides comprehensive security checklist and patterns.
---
# Security Review Skill
This skill ensures all code follows security best practices and identifies potential vulnerabilities.
## When to Activate
- Implementing authentication or authorization
- Handling user input or file uploads
- Creating new API endpoints
- Working with secrets or credentials
- Implementing payment features
- Storing or transmitting sensitive data
- Integrating third-party APIs
## Security Checklist
### 1. Secrets Management
#### ❌ NEVER Do This
```typescript
const apiKey = "sk-proj-xxxxx" // Hardcoded secret
const dbPassword = "password123" // In source code
```
#### ✅ ALWAYS Do This
```typescript
const apiKey = process.env.OPENAI_API_KEY
const dbUrl = process.env.DATABASE_URL
// Verify secrets exist
if (!apiKey) {
throw new Error('OPENAI_API_KEY not configured')
}
```
#### Verification Steps
- [ ] No hardcoded API keys, tokens, or passwords
- [ ] All secrets in environment variables
- [ ] `.env.local` in .gitignore
- [ ] No secrets in git history
- [ ] Production secrets in hosting platform (Vercel, Railway)
### 2. Input Validation
#### Always Validate User Input
```typescript
import { z } from 'zod'
// Define validation schema
const CreateUserSchema = z.object({
email: z.string().email(),
name: z.string().min(1).max(100),
age: z.number().int().min(0).max(150)
})
// Validate before processing
export async function createUser(input: unknown) {
try {
const validated = CreateUserSchema.parse(input)
return await db.users.create(validated)
} catch (error) {
if (error instanceof z.ZodError) {
return { success: false, errors: error.errors }
}
throw error
}
}
```
#### File Upload Validation
```typescript
function validateFileUpload(file: File) {
// Size check (5MB max)
const maxSize = 5 * 1024 * 1024
if (file.size > maxSize) {
throw new Error('File too large (max 5MB)')
}
// Type check
const allowedTypes = ['image/jpeg', 'image/png', 'image/gif']
if (!allowedTypes.includes(file.type)) {
throw new Error('Invalid file type')
}
// Extension check
const allowedExtensions = ['.jpg', '.jpeg', '.png', '.gif']
const extension = file.name.toLowerCase().match(/\.[^.]+$/)?.[0]
if (!extension || !allowedExtensions.includes(extension)) {
throw new Error('Invalid file extension')
}
return true
}
```
#### Verification Steps
- [ ] All user inputs validated with schemas
- [ ] File uploads restricted (size, type, extension)
- [ ] No direct use of user input in queries
- [ ] Whitelist validation (not blacklist)
- [ ] Error messages don't leak sensitive info
### 3. SQL Injection Prevention
#### ❌ NEVER Concatenate SQL
```typescript
// DANGEROUS - SQL Injection vulnerability
const query = `SELECT * FROM users WHERE email = '${userEmail}'`
await db.query(query)
```
#### ✅ ALWAYS Use Parameterized Queries
```typescript
// Safe - parameterized query
const { data } = await supabase
.from('users')
.select('*')
.eq('email', userEmail)
// Or with raw SQL
await db.query(
'SELECT * FROM users WHERE email = $1',
[userEmail]
)
```
#### Verification Steps
- [ ] All database queries use parameterized queries
- [ ] No string concatenation in SQL
- [ ] ORM/query builder used correctly
- [ ] Supabase queries properly sanitized
### 4. Authentication & Authorization
#### JWT Token Handling
```typescript
// ❌ WRONG: localStorage (vulnerable to XSS)
localStorage.setItem('token', token)
// ✅ CORRECT: httpOnly cookies
res.setHeader('Set-Cookie',
`token=${token}; HttpOnly; Secure; SameSite=Strict; Max-Age=3600`)
```
#### Authorization Checks
```typescript
export async function deleteUser(userId: string, requesterId: string) {
// ALWAYS verify authorization first
const requester = await db.users.findUnique({
where: { id: requesterId }
})
if (requester.role !== 'admin') {
return NextResponse.json(
{ error: 'Unauthorized' },
{ status: 403 }
)
}
// Proceed with deletion
await db.users.delete({ where: { id: userId } })
}
```
#### Row Level Security (Supabase)
```sql
-- Enable RLS on all tables
ALTER TABLE users ENABLE ROW LEVEL SECURITY;
-- Users can only view their own data
CREATE POLICY "Users view own data"
ON users FOR SELECT
USING (auth.uid() = id);
-- Users can only update their own data
CREATE POLICY "Users update own data"
ON users FOR UPDATE
USING (auth.uid() = id);
```
#### Verification Steps
- [ ] Tokens stored in httpOnly cookies (not localStorage)
- [ ] Authorization checks before sensitive operations
- [ ] Row Level Security enabled in Supabase
- [ ] Role-based access control implemented
- [ ] Session management secure
### 5. XSS Prevention
#### Sanitize HTML
```typescript
import DOMPurify from 'isomorphic-dompurify'
// ALWAYS sanitize user-provided HTML
function renderUserContent(html: string) {
const clean = DOMPurify.sanitize(html, {
ALLOWED_TAGS: ['b', 'i', 'em', 'strong', 'p'],
ALLOWED_ATTR: []
})
return <div dangerouslySetInnerHTML={{ __html: clean }} />
}
```
#### Content Security Policy
```typescript
// next.config.js
const securityHeaders = [
{
key: 'Content-Security-Policy',
value: `
default-src 'self';
script-src 'self' 'unsafe-eval' 'unsafe-inline';
style-src 'self' 'unsafe-inline';
img-src 'self' data: https:;
font-src 'self';
connect-src 'self' https://api.example.com;
`.replace(/\s{2,}/g, ' ').trim()
}
]
```
#### Verification Steps
- [ ] User-provided HTML sanitized
- [ ] CSP headers configured
- [ ] No unvalidated dynamic content rendering
- [ ] React's built-in XSS protection used
### 6. CSRF Protection
#### CSRF Tokens
```typescript
import { csrf } from '@/lib/csrf'
export async function POST(request: Request) {
const token = request.headers.get('X-CSRF-Token')
if (!csrf.verify(token)) {
return NextResponse.json(
{ error: 'Invalid CSRF token' },
{ status: 403 }
)
}
// Process request
}
```
#### SameSite Cookies
```typescript
res.setHeader('Set-Cookie',
`session=${sessionId}; HttpOnly; Secure; SameSite=Strict`)
```
#### Verification Steps
- [ ] CSRF tokens on state-changing operations
- [ ] SameSite=Strict on all cookies
- [ ] Double-submit cookie pattern implemented
### 7. Rate Limiting
#### API Rate Limiting
```typescript
import rateLimit from 'express-rate-limit'
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // 100 requests per window
message: 'Too many requests'
})
// Apply to routes
app.use('/api/', limiter)
```
#### Expensive Operations
```typescript
// Aggressive rate limiting for searches
const searchLimiter = rateLimit({
windowMs: 60 * 1000, // 1 minute
max: 10, // 10 requests per minute
message: 'Too many search requests'
})
app.use('/api/search', searchLimiter)
```
#### Verification Steps
- [ ] Rate limiting on all API endpoints
- [ ] Stricter limits on expensive operations
- [ ] IP-based rate limiting
- [ ] User-based rate limiting (authenticated)
### 8. Sensitive Data Exposure
#### Logging
```typescript
// ❌ WRONG: Logging sensitive data
console.log('User login:', { email, password })
console.log('Payment:', { cardNumber, cvv })
// ✅ CORRECT: Redact sensitive data
console.log('User login:', { email, userId })
console.log('Payment:', { last4: card.last4, userId })
```
#### Error Messages
```typescript
// ❌ WRONG: Exposing internal details
catch (error) {
return NextResponse.json(
{ error: error.message, stack: error.stack },
{ status: 500 }
)
}
// ✅ CORRECT: Generic error messages
catch (error) {
console.error('Internal error:', error)
return NextResponse.json(
{ error: 'An error occurred. Please try again.' },
{ status: 500 }
)
}
```
#### Verification Steps
- [ ] No passwords, tokens, or secrets in logs
- [ ] Error messages generic for users
- [ ] Detailed errors only in server logs
- [ ] No stack traces exposed to users
### 9. Blockchain Security (Solana)
#### Wallet Verification
```typescript
import { verify } from '@solana/web3.js'
async function verifyWalletOwnership(
publicKey: string,
signature: string,
message: string
) {
try {
const isValid = verify(
Buffer.from(message),
Buffer.from(signature, 'base64'),
Buffer.from(publicKey, 'base64')
)
return isValid
} catch (error) {
return false
}
}
```
#### Transaction Verification
```typescript
async function verifyTransaction(transaction: Transaction) {
// Verify recipient
if (transaction.to !== expectedRecipient) {
throw new Error('Invalid recipient')
}
// Verify amount
if (transaction.amount > maxAmount) {
throw new Error('Amount exceeds limit')
}
// Verify user has sufficient balance
const balance = await getBalance(transaction.from)
if (balance < transaction.amount) {
throw new Error('Insufficient balance')
}
return true
}
```
#### Verification Steps
- [ ] Wallet signatures verified
- [ ] Transaction details validated
- [ ] Balance checks before transactions
- [ ] No blind transaction signing
### 10. Dependency Security
#### Regular Updates
```bash
# Check for vulnerabilities
npm audit
# Fix automatically fixable issues
npm audit fix
# Update dependencies
npm update
# Check for outdated packages
npm outdated
```
#### Lock Files
```bash
# ALWAYS commit lock files
git add package-lock.json
# Use in CI/CD for reproducible builds
npm ci # Instead of npm install
```
#### Verification Steps
- [ ] Dependencies up to date
- [ ] No known vulnerabilities (npm audit clean)
- [ ] Lock files committed
- [ ] Dependabot enabled on GitHub
- [ ] Regular security updates
## Security Testing
### Automated Security Tests
```typescript
// Test authentication
test('requires authentication', async () => {
const response = await fetch('/api/protected')
expect(response.status).toBe(401)
})
// Test authorization
test('requires admin role', async () => {
const response = await fetch('/api/admin', {
headers: { Authorization: `Bearer ${userToken}` }
})
expect(response.status).toBe(403)
})
// Test input validation
test('rejects invalid input', async () => {
const response = await fetch('/api/users', {
method: 'POST',
body: JSON.stringify({ email: 'not-an-email' })
})
expect(response.status).toBe(400)
})
// Test rate limiting
test('enforces rate limits', async () => {
const requests = Array(101).fill(null).map(() =>
fetch('/api/endpoint')
)
const responses = await Promise.all(requests)
const tooManyRequests = responses.filter(r => r.status === 429)
expect(tooManyRequests.length).toBeGreaterThan(0)
})
```
## Pre-Deployment Security Checklist
Before ANY production deployment:
- [ ] **Secrets**: No hardcoded secrets, all in env vars
- [ ] **Input Validation**: All user inputs validated
- [ ] **SQL Injection**: All queries parameterized
- [ ] **XSS**: User content sanitized
- [ ] **CSRF**: Protection enabled
- [ ] **Authentication**: Proper token handling
- [ ] **Authorization**: Role checks in place
- [ ] **Rate Limiting**: Enabled on all endpoints
- [ ] **HTTPS**: Enforced in production
- [ ] **Security Headers**: CSP, X-Frame-Options configured
- [ ] **Error Handling**: No sensitive data in errors
- [ ] **Logging**: No sensitive data logged
- [ ] **Dependencies**: Up to date, no vulnerabilities
- [ ] **Row Level Security**: Enabled in Supabase
- [ ] **CORS**: Properly configured
- [ ] **File Uploads**: Validated (size, type)
- [ ] **Wallet Signatures**: Verified (if blockchain)
## Resources
- [OWASP Top 10](https://owasp.org/www-project-top-ten/)
- [Next.js Security](https://nextjs.org/docs/security)
- [Supabase Security](https://supabase.com/docs/guides/auth)
- [Web Security Academy](https://portswigger.net/web-security)
---
**Remember**: Security is not optional. One vulnerability can compromise the entire platform. When in doubt, err on the side of caution.

361
skills/security-review/cloud-infrastructure-security.md Normal file
View File

@@ -0,0 +1,361 @@
| name | description |
|------|-------------|
| cloud-infrastructure-security | Use this skill when deploying to cloud platforms, configuring infrastructure, managing IAM policies, setting up logging/monitoring, or implementing CI/CD pipelines. Provides cloud security checklist aligned with best practices. |
# Cloud & Infrastructure Security Skill
This skill ensures cloud infrastructure, CI/CD pipelines, and deployment configurations follow security best practices and comply with industry standards.
## When to Activate
- Deploying applications to cloud platforms (AWS, Vercel, Railway, Cloudflare)
- Configuring IAM roles and permissions
- Setting up CI/CD pipelines
- Implementing infrastructure as code (Terraform, CloudFormation)
- Configuring logging and monitoring
- Managing secrets in cloud environments
- Setting up CDN and edge security
- Implementing disaster recovery and backup strategies
## Cloud Security Checklist
### 1. IAM & Access Control
#### Principle of Least Privilege
```yaml
# ✅ CORRECT: Minimal permissions
iam_role:
permissions:
- s3:GetObject # Only read access
- s3:ListBucket
resources:
- arn:aws:s3:::my-bucket/* # Specific bucket only
# ❌ WRONG: Overly broad permissions
iam_role:
permissions:
- s3:* # All S3 actions
resources:
- "*" # All resources
```
#### Multi-Factor Authentication (MFA)
```bash
# ALWAYS enable MFA for root/admin accounts
aws iam enable-mfa-device \
--user-name admin \
--serial-number arn:aws:iam::123456789:mfa/admin \
--authentication-code1 123456 \
--authentication-code2 789012
```
#### Verification Steps
- [ ] No root account usage in production
- [ ] MFA enabled for all privileged accounts
- [ ] Service accounts use roles, not long-lived credentials
- [ ] IAM policies follow least privilege
- [ ] Regular access reviews conducted
- [ ] Unused credentials rotated or removed
### 2. Secrets Management
#### Cloud Secrets Managers
```typescript
// ✅ CORRECT: Use cloud secrets manager
import { SecretsManager } from '@aws-sdk/client-secrets-manager';
const client = new SecretsManager({ region: 'us-east-1' });
const secret = await client.getSecretValue({ SecretId: 'prod/api-key' });
const apiKey = JSON.parse(secret.SecretString).key;
// ❌ WRONG: Hardcoded or in environment variables only
const apiKey = process.env.API_KEY; // Not rotated, not audited
```
#### Secrets Rotation
```bash
# Set up automatic rotation for database credentials
aws secretsmanager rotate-secret \
--secret-id prod/db-password \
--rotation-lambda-arn arn:aws:lambda:region:account:function:rotate \
--rotation-rules AutomaticallyAfterDays=30
```
#### Verification Steps
- [ ] All secrets stored in cloud secrets manager (AWS Secrets Manager, Vercel Secrets)
- [ ] Automatic rotation enabled for database credentials
- [ ] API keys rotated at least quarterly
- [ ] No secrets in code, logs, or error messages
- [ ] Audit logging enabled for secret access
### 3. Network Security
#### VPC and Firewall Configuration
```terraform
# ✅ CORRECT: Restricted security group
resource "aws_security_group" "app" {
name = "app-sg"
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["10.0.0.0/16"] # Internal VPC only
}
egress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Only HTTPS outbound
}
}
# ❌ WRONG: Open to the internet
resource "aws_security_group" "bad" {
ingress {
from_port = 0
to_port = 65535
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # All ports, all IPs!
}
}
```
#### Verification Steps
- [ ] Database not publicly accessible
- [ ] SSH/RDP ports restricted to VPN/bastion only
- [ ] Security groups follow least privilege
- [ ] Network ACLs configured
- [ ] VPC flow logs enabled
### 4. Logging & Monitoring
#### CloudWatch/Logging Configuration
```typescript
// ✅ CORRECT: Comprehensive logging
import { CloudWatchLogsClient, CreateLogStreamCommand } from '@aws-sdk/client-cloudwatch-logs';
const logSecurityEvent = async (event: SecurityEvent) => {
await cloudwatch.putLogEvents({
logGroupName: '/aws/security/events',
logStreamName: 'authentication',
logEvents: [{
timestamp: Date.now(),
message: JSON.stringify({
type: event.type,
userId: event.userId,
ip: event.ip,
result: event.result,
// Never log sensitive data
})
}]
});
};
```
#### Verification Steps
- [ ] CloudWatch/logging enabled for all services
- [ ] Failed authentication attempts logged
- [ ] Admin actions audited
- [ ] Log retention configured (90+ days for compliance)
- [ ] Alerts configured for suspicious activity
- [ ] Logs centralized and tamper-proof
### 5. CI/CD Pipeline Security
#### Secure Pipeline Configuration
```yaml
# ✅ CORRECT: Secure GitHub Actions workflow
name: Deploy
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
permissions:
contents: read # Minimal permissions
steps:
- uses: actions/checkout@v4
# Scan for secrets
- name: Secret scanning
uses: trufflesecurity/trufflehog@main
# Dependency audit
- name: Audit dependencies
run: npm audit --audit-level=high
# Use OIDC, not long-lived tokens
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789:role/GitHubActionsRole
aws-region: us-east-1
```
#### Supply Chain Security
```json
// package.json - Use lock files and integrity checks
{
"scripts": {
"install": "npm ci", // Use ci for reproducible builds
"audit": "npm audit --audit-level=moderate",
"check": "npm outdated"
}
}
```
#### Verification Steps
- [ ] OIDC used instead of long-lived credentials
- [ ] Secrets scanning in pipeline
- [ ] Dependency vulnerability scanning
- [ ] Container image scanning (if applicable)
- [ ] Branch protection rules enforced
- [ ] Code review required before merge
- [ ] Signed commits enforced
### 6. Cloudflare & CDN Security
#### Cloudflare Security Configuration
```typescript
// ✅ CORRECT: Cloudflare Workers with security headers
export default {
async fetch(request: Request): Promise<Response> {
const response = await fetch(request);
// Add security headers
const headers = new Headers(response.headers);
headers.set('X-Frame-Options', 'DENY');
headers.set('X-Content-Type-Options', 'nosniff');
headers.set('Referrer-Policy', 'strict-origin-when-cross-origin');
headers.set('Permissions-Policy', 'geolocation=(), microphone=()');
return new Response(response.body, {
status: response.status,
headers
});
}
};
```
#### WAF Rules
```bash
# Enable Cloudflare WAF managed rules
# - OWASP Core Ruleset
# - Cloudflare Managed Ruleset
# - Rate limiting rules
# - Bot protection
```
#### Verification Steps
- [ ] WAF enabled with OWASP rules
- [ ] Rate limiting configured
- [ ] Bot protection active
- [ ] DDoS protection enabled
- [ ] Security headers configured
- [ ] SSL/TLS strict mode enabled
### 7. Backup & Disaster Recovery
#### Automated Backups
```terraform
# ✅ CORRECT: Automated RDS backups
resource "aws_db_instance" "main" {
allocated_storage = 20
engine = "postgres"
backup_retention_period = 30 # 30 days retention
backup_window = "03:00-04:00"
maintenance_window = "mon:04:00-mon:05:00"
enabled_cloudwatch_logs_exports = ["postgresql"]
deletion_protection = true # Prevent accidental deletion
}
```
#### Verification Steps
- [ ] Automated daily backups configured
- [ ] Backup retention meets compliance requirements
- [ ] Point-in-time recovery enabled
- [ ] Backup testing performed quarterly
- [ ] Disaster recovery plan documented
- [ ] RPO and RTO defined and tested
## Pre-Deployment Cloud Security Checklist
Before ANY production cloud deployment:
- [ ] **IAM**: Root account not used, MFA enabled, least privilege policies
- [ ] **Secrets**: All secrets in cloud secrets manager with rotation
- [ ] **Network**: Security groups restricted, no public databases
- [ ] **Logging**: CloudWatch/logging enabled with retention
- [ ] **Monitoring**: Alerts configured for anomalies
- [ ] **CI/CD**: OIDC auth, secrets scanning, dependency audits
- [ ] **CDN/WAF**: Cloudflare WAF enabled with OWASP rules
- [ ] **Encryption**: Data encrypted at rest and in transit
- [ ] **Backups**: Automated backups with tested recovery
- [ ] **Compliance**: GDPR/HIPAA requirements met (if applicable)
- [ ] **Documentation**: Infrastructure documented, runbooks created
- [ ] **Incident Response**: Security incident plan in place
## Common Cloud Security Misconfigurations
### S3 Bucket Exposure
```bash
# ❌ WRONG: Public bucket
aws s3api put-bucket-acl --bucket my-bucket --acl public-read
# ✅ CORRECT: Private bucket with specific access
aws s3api put-bucket-acl --bucket my-bucket --acl private
aws s3api put-bucket-policy --bucket my-bucket --policy file://policy.json
```
### RDS Public Access
```terraform
# ❌ WRONG
resource "aws_db_instance" "bad" {
publicly_accessible = true # NEVER do this!
}
# ✅ CORRECT
resource "aws_db_instance" "good" {
publicly_accessible = false
vpc_security_group_ids = [aws_security_group.db.id]
}
```
## Resources
- [AWS Security Best Practices](https://aws.amazon.com/security/best-practices/)
- [CIS AWS Foundations Benchmark](https://www.cisecurity.org/benchmark/amazon_web_services)
- [Cloudflare Security Documentation](https://developers.cloudflare.com/security/)
- [OWASP Cloud Security](https://owasp.org/www-project-cloud-security/)
- [Terraform Security Best Practices](https://www.terraform.io/docs/cloud/guides/recommended-practices/)
**Remember**: Cloud misconfigurations are the leading cause of data breaches. A single exposed S3 bucket or overly permissive IAM policy can compromise your entire infrastructure. Always follow the principle of least privilege and defense in depth.

164
skills/security-scan/SKILL.md Normal file
View File

@@ -0,0 +1,164 @@
---
name: security-scan
description: Scan your Claude Code configuration (.claude/ directory) for security vulnerabilities, misconfigurations, and injection risks using AgentShield. Checks CLAUDE.md, settings.json, MCP servers, hooks, and agent definitions.
---
# Security Scan Skill
Audit your Claude Code configuration for security issues using [AgentShield](https://github.com/affaan-m/agentshield).
## When to Activate
- Setting up a new Claude Code project
- After modifying `.claude/settings.json`, `CLAUDE.md`, or MCP configs
- Before committing configuration changes
- When onboarding to a new repository with existing Claude Code configs
- Periodic security hygiene checks
## What It Scans
| File | Checks |
|------|--------|
| `CLAUDE.md` | Hardcoded secrets, auto-run instructions, prompt injection patterns |
| `settings.json` | Overly permissive allow lists, missing deny lists, dangerous bypass flags |
| `mcp.json` | Risky MCP servers, hardcoded env secrets, npx supply chain risks |
| `hooks/` | Command injection via interpolation, data exfiltration, silent error suppression |
| `agents/*.md` | Unrestricted tool access, prompt injection surface, missing model specs |
## Prerequisites
AgentShield must be installed. Check and install if needed:
```bash
# Check if installed
npx ecc-agentshield --version
# Install globally (recommended)
npm install -g ecc-agentshield
# Or run directly via npx (no install needed)
npx ecc-agentshield scan .
```
## Usage
### Basic Scan
Run against the current project's `.claude/` directory:
```bash
# Scan current project
npx ecc-agentshield scan
# Scan a specific path
npx ecc-agentshield scan --path /path/to/.claude
# Scan with minimum severity filter
npx ecc-agentshield scan --min-severity medium
```
### Output Formats
```bash
# Terminal output (default) — colored report with grade
npx ecc-agentshield scan
# JSON — for CI/CD integration
npx ecc-agentshield scan --format json
# Markdown — for documentation
npx ecc-agentshield scan --format markdown
# HTML — self-contained dark-theme report
npx ecc-agentshield scan --format html > security-report.html
```
### Auto-Fix
Apply safe fixes automatically (only fixes marked as auto-fixable):
```bash
npx ecc-agentshield scan --fix
```
This will:
- Replace hardcoded secrets with environment variable references
- Tighten wildcard permissions to scoped alternatives
- Never modify manual-only suggestions
### Opus 4.6 Deep Analysis
Run the adversarial three-agent pipeline for deeper analysis:
```bash
# Requires ANTHROPIC_API_KEY
export ANTHROPIC_API_KEY=your-key
npx ecc-agentshield scan --opus --stream
```
This runs:
1. **Attacker (Red Team)** — finds attack vectors
2. **Defender (Blue Team)** — recommends hardening
3. **Auditor (Final Verdict)** — synthesizes both perspectives
### Initialize Secure Config
Scaffold a new secure `.claude/` configuration from scratch:
```bash
npx ecc-agentshield init
```
Creates:
- `settings.json` with scoped permissions and deny list
- `CLAUDE.md` with security best practices
- `mcp.json` placeholder
### GitHub Action
Add to your CI pipeline:
```yaml
- uses: affaan-m/agentshield@v1
with:
path: '.'
min-severity: 'medium'
fail-on-findings: true
```
## Severity Levels
| Grade | Score | Meaning |
|-------|-------|---------|
| A | 90-100 | Secure configuration |
| B | 75-89 | Minor issues |
| C | 60-74 | Needs attention |
| D | 40-59 | Significant risks |
| F | 0-39 | Critical vulnerabilities |
## Interpreting Results
### Critical Findings (fix immediately)
- Hardcoded API keys or tokens in config files
- `Bash(*)` in the allow list (unrestricted shell access)
- Command injection in hooks via `${file}` interpolation
- Shell-running MCP servers
### High Findings (fix before production)
- Auto-run instructions in CLAUDE.md (prompt injection vector)
- Missing deny lists in permissions
- Agents with unnecessary Bash access
### Medium Findings (recommended)
- Silent error suppression in hooks (`2>/dev/null`, `|| true`)
- Missing PreToolUse security hooks
- `npx -y` auto-install in MCP server configs
### Info Findings (awareness)
- Missing descriptions on MCP servers
- Prohibitive instructions correctly flagged as good practice
## Links
- **GitHub**: [github.com/affaan-m/agentshield](https://github.com/affaan-m/agentshield)
- **npm**: [npmjs.com/package/ecc-agentshield](https://www.npmjs.com/package/ecc-agentshield)

313
skills/springboot-patterns/SKILL.md Normal file
View File

@@ -0,0 +1,313 @@
---
name: springboot-patterns
description: Spring Boot architecture patterns, REST API design, layered services, data access, caching, async processing, and logging. Use for Java Spring Boot backend work.
---
# Spring Boot Development Patterns
Spring Boot architecture and API patterns for scalable, production-grade services.
## When to Activate
- Building REST APIs with Spring MVC or WebFlux
- Structuring controller → service → repository layers
- Configuring Spring Data JPA, caching, or async processing
- Adding validation, exception handling, or pagination
- Setting up profiles for dev/staging/production environments
- Implementing event-driven patterns with Spring Events or Kafka
## REST API Structure
```java
@RestController
@RequestMapping("/api/markets")
@Validated
class MarketController {
private final MarketService marketService;
MarketController(MarketService marketService) {
this.marketService = marketService;
}
@GetMapping
ResponseEntity<Page<MarketResponse>> list(
@RequestParam(defaultValue = "0") int page,
@RequestParam(defaultValue = "20") int size) {
Page<Market> markets = marketService.list(PageRequest.of(page, size));
return ResponseEntity.ok(markets.map(MarketResponse::from));
}
@PostMapping
ResponseEntity<MarketResponse> create(@Valid @RequestBody CreateMarketRequest request) {
Market market = marketService.create(request);
return ResponseEntity.status(HttpStatus.CREATED).body(MarketResponse.from(market));
}
}
```
## Repository Pattern (Spring Data JPA)
```java
public interface MarketRepository extends JpaRepository<MarketEntity, Long> {
@Query("select m from MarketEntity m where m.status = :status order by m.volume desc")
List<MarketEntity> findActive(@Param("status") MarketStatus status, Pageable pageable);
}
```
## Service Layer with Transactions
```java
@Service
public class MarketService {
private final MarketRepository repo;
public MarketService(MarketRepository repo) {
this.repo = repo;
}
@Transactional
public Market create(CreateMarketRequest request) {
MarketEntity entity = MarketEntity.from(request);
MarketEntity saved = repo.save(entity);
return Market.from(saved);
}
}
```
## DTOs and Validation
```java
public record CreateMarketRequest(
@NotBlank @Size(max = 200) String name,
@NotBlank @Size(max = 2000) String description,
@NotNull @FutureOrPresent Instant endDate,
@NotEmpty List<@NotBlank String> categories) {}
public record MarketResponse(Long id, String name, MarketStatus status) {
static MarketResponse from(Market market) {
return new MarketResponse(market.id(), market.name(), market.status());
}
}
```
## Exception Handling
```java
@ControllerAdvice
class GlobalExceptionHandler {
@ExceptionHandler(MethodArgumentNotValidException.class)
ResponseEntity<ApiError> handleValidation(MethodArgumentNotValidException ex) {
String message = ex.getBindingResult().getFieldErrors().stream()
.map(e -> e.getField() + ": " + e.getDefaultMessage())
.collect(Collectors.joining(", "));
return ResponseEntity.badRequest().body(ApiError.validation(message));
}
@ExceptionHandler(AccessDeniedException.class)
ResponseEntity<ApiError> handleAccessDenied() {
return ResponseEntity.status(HttpStatus.FORBIDDEN).body(ApiError.of("Forbidden"));
}
@ExceptionHandler(Exception.class)
ResponseEntity<ApiError> handleGeneric(Exception ex) {
// Log unexpected errors with stack traces
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(ApiError.of("Internal server error"));
}
}
```
## Caching
Requires `@EnableCaching` on a configuration class.
```java
@Service
public class MarketCacheService {
private final MarketRepository repo;
public MarketCacheService(MarketRepository repo) {
this.repo = repo;
}
@Cacheable(value = "market", key = "#id")
public Market getById(Long id) {
return repo.findById(id)
.map(Market::from)
.orElseThrow(() -> new EntityNotFoundException("Market not found"));
}
@CacheEvict(value = "market", key = "#id")
public void evict(Long id) {}
}
```
## Async Processing
Requires `@EnableAsync` on a configuration class.
```java
@Service
public class NotificationService {
@Async
public CompletableFuture<Void> sendAsync(Notification notification) {
// send email/SMS
return CompletableFuture.completedFuture(null);
}
}
```
## Logging (SLF4J)
```java
@Service
public class ReportService {
private static final Logger log = LoggerFactory.getLogger(ReportService.class);
public Report generate(Long marketId) {
log.info("generate_report marketId={}", marketId);
try {
// logic
} catch (Exception ex) {
log.error("generate_report_failed marketId={}", marketId, ex);
throw ex;
}
return new Report();
}
}
```
## Middleware / Filters
```java
@Component
public class RequestLoggingFilter extends OncePerRequestFilter {
private static final Logger log = LoggerFactory.getLogger(RequestLoggingFilter.class);
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain filterChain) throws ServletException, IOException {
long start = System.currentTimeMillis();
try {
filterChain.doFilter(request, response);
} finally {
long duration = System.currentTimeMillis() - start;
log.info("req method={} uri={} status={} durationMs={}",
request.getMethod(), request.getRequestURI(), response.getStatus(), duration);
}
}
}
```
## Pagination and Sorting
```java
PageRequest page = PageRequest.of(pageNumber, pageSize, Sort.by("createdAt").descending());
Page<Market> results = marketService.list(page);
```
## Error-Resilient External Calls
```java
public <T> T withRetry(Supplier<T> supplier, int maxRetries) {
int attempts = 0;
while (true) {
try {
return supplier.get();
} catch (Exception ex) {
attempts++;
if (attempts >= maxRetries) {
throw ex;
}
try {
Thread.sleep((long) Math.pow(2, attempts) * 100L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw ex;
}
}
}
}
```
## Rate Limiting (Filter + Bucket4j)
**Security Note**: The `X-Forwarded-For` header is untrusted by default because clients can spoof it.
Only use forwarded headers when:
1. Your app is behind a trusted reverse proxy (nginx, AWS ALB, etc.)
2. You have registered `ForwardedHeaderFilter` as a bean
3. You have configured `server.forward-headers-strategy=NATIVE` or `FRAMEWORK` in application properties
4. Your proxy is configured to overwrite (not append to) the `X-Forwarded-For` header
When `ForwardedHeaderFilter` is properly configured, `request.getRemoteAddr()` will automatically
return the correct client IP from the forwarded headers. Without this configuration, use
`request.getRemoteAddr()` directly—it returns the immediate connection IP, which is the only
trustworthy value.
```java
@Component
public class RateLimitFilter extends OncePerRequestFilter {
private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();
/*
* SECURITY: This filter uses request.getRemoteAddr() to identify clients for rate limiting.
*
* If your application is behind a reverse proxy (nginx, AWS ALB, etc.), you MUST configure
* Spring to handle forwarded headers properly for accurate client IP detection:
*
* 1. Set server.forward-headers-strategy=NATIVE (for cloud platforms) or FRAMEWORK in
* application.properties/yaml
* 2. If using FRAMEWORK strategy, register ForwardedHeaderFilter:
*
* @Bean
* ForwardedHeaderFilter forwardedHeaderFilter() {
* return new ForwardedHeaderFilter();
* }
*
* 3. Ensure your proxy overwrites (not appends) the X-Forwarded-For header to prevent spoofing
* 4. Configure server.tomcat.remoteip.trusted-proxies or equivalent for your container
*
* Without this configuration, request.getRemoteAddr() returns the proxy IP, not the client IP.
* Do NOT read X-Forwarded-For directly—it is trivially spoofable without trusted proxy handling.
*/
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain filterChain) throws ServletException, IOException {
// Use getRemoteAddr() which returns the correct client IP when ForwardedHeaderFilter
// is configured, or the direct connection IP otherwise. Never trust X-Forwarded-For
// headers directly without proper proxy configuration.
String clientIp = request.getRemoteAddr();
Bucket bucket = buckets.computeIfAbsent(clientIp,
k -> Bucket.builder()
.addLimit(Bandwidth.classic(100, Refill.greedy(100, Duration.ofMinutes(1))))
.build());
if (bucket.tryConsume(1)) {
filterChain.doFilter(request, response);
} else {
response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
}
}
}
```
## Background Jobs
Use Spring’s `@Scheduled` or integrate with queues (e.g., Kafka, SQS, RabbitMQ). Keep handlers idempotent and observable.
## Observability
- Structured logging (JSON) via Logback encoder
- Metrics: Micrometer + Prometheus/OTel
- Tracing: Micrometer Tracing with OpenTelemetry or Brave backend
## Production Defaults
- Prefer constructor injection, avoid field injection
- Enable `spring.mvc.problemdetails.enabled=true` for RFC 7807 errors (Spring Boot 3+)
- Configure HikariCP pool sizes for workload, set timeouts
- Use `@Transactional(readOnly = true)` for queries
- Enforce null-safety via `@NonNull` and `Optional` where appropriate
**Remember**: Keep controllers thin, services focused, repositories simple, and errors handled centrally. Optimize for maintainability and testability.

271
skills/springboot-security/SKILL.md Normal file
View File

@@ -0,0 +1,271 @@
---
name: springboot-security
description: Spring Security best practices for authn/authz, validation, CSRF, secrets, headers, rate limiting, and dependency security in Java Spring Boot services.
---
# Spring Boot Security Review
Use when adding auth, handling input, creating endpoints, or dealing with secrets.
## When to Activate
- Adding authentication (JWT, OAuth2, session-based)
- Implementing authorization (@PreAuthorize, role-based access)
- Validating user input (Bean Validation, custom validators)
- Configuring CORS, CSRF, or security headers
- Managing secrets (Vault, environment variables)
- Adding rate limiting or brute-force protection
- Scanning dependencies for CVEs
## Authentication
- Prefer stateless JWT or opaque tokens with revocation list
- Use `httpOnly`, `Secure`, `SameSite=Strict` cookies for sessions
- Validate tokens with `OncePerRequestFilter` or resource server
```java
@Component
public class JwtAuthFilter extends OncePerRequestFilter {
private final JwtService jwtService;
public JwtAuthFilter(JwtService jwtService) {
this.jwtService = jwtService;
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain chain) throws ServletException, IOException {
String header = request.getHeader(HttpHeaders.AUTHORIZATION);
if (header != null && header.startsWith("Bearer ")) {
String token = header.substring(7);
Authentication auth = jwtService.authenticate(token);
SecurityContextHolder.getContext().setAuthentication(auth);
}
chain.doFilter(request, response);
}
}
```
## Authorization
- Enable method security: `@EnableMethodSecurity`
- Use `@PreAuthorize("hasRole('ADMIN')")` or `@PreAuthorize("@authz.canEdit(#id)")`
- Deny by default; expose only required scopes
```java
@RestController
@RequestMapping("/api/admin")
public class AdminController {
@PreAuthorize("hasRole('ADMIN')")
@GetMapping("/users")
public List<UserDto> listUsers() {
return userService.findAll();
}
@PreAuthorize("@authz.isOwner(#id, authentication)")
@DeleteMapping("/users/{id}")
public ResponseEntity<Void> deleteUser(@PathVariable Long id) {
userService.delete(id);
return ResponseEntity.noContent().build();
}
}
```
## Input Validation
- Use Bean Validation with `@Valid` on controllers
- Apply constraints on DTOs: `@NotBlank`, `@Email`, `@Size`, custom validators
- Sanitize any HTML with a whitelist before rendering
```java
// BAD: No validation
@PostMapping("/users")
public User createUser(@RequestBody UserDto dto) {
return userService.create(dto);
}
// GOOD: Validated DTO
public record CreateUserDto(
@NotBlank @Size(max = 100) String name,
@NotBlank @Email String email,
@NotNull @Min(0) @Max(150) Integer age
) {}
@PostMapping("/users")
public ResponseEntity<UserDto> createUser(@Valid @RequestBody CreateUserDto dto) {
return ResponseEntity.status(HttpStatus.CREATED)
.body(userService.create(dto));
}
```
## SQL Injection Prevention
- Use Spring Data repositories or parameterized queries
- For native queries, use `:param` bindings; never concatenate strings
```java
// BAD: String concatenation in native query
@Query(value = "SELECT * FROM users WHERE name = '" + name + "'", nativeQuery = true)
// GOOD: Parameterized native query
@Query(value = "SELECT * FROM users WHERE name = :name", nativeQuery = true)
List<User> findByName(@Param("name") String name);
// GOOD: Spring Data derived query (auto-parameterized)
List<User> findByEmailAndActiveTrue(String email);
```
## Password Encoding
- Always hash passwords with BCrypt or Argon2 — never store plaintext
- Use `PasswordEncoder` bean, not manual hashing
```java
@Bean
public PasswordEncoder passwordEncoder() {
return new BCryptPasswordEncoder(12); // cost factor 12
}
// In service
public User register(CreateUserDto dto) {
String hashedPassword = passwordEncoder.encode(dto.password());
return userRepository.save(new User(dto.email(), hashedPassword));
}
```
## CSRF Protection
- For browser session apps, keep CSRF enabled; include token in forms/headers
- For pure APIs with Bearer tokens, disable CSRF and rely on stateless auth
```java
http
.csrf(csrf -> csrf.disable())
.sessionManagement(sm -> sm.sessionCreationPolicy(SessionCreationPolicy.STATELESS));
```
## Secrets Management
- No secrets in source; load from env or vault
- Keep `application.yml` free of credentials; use placeholders
- Rotate tokens and DB credentials regularly
```yaml
# BAD: Hardcoded in application.yml
spring:
datasource:
password: mySecretPassword123
# GOOD: Environment variable placeholder
spring:
datasource:
password: ${DB_PASSWORD}
# GOOD: Spring Cloud Vault integration
spring:
cloud:
vault:
uri: https://vault.example.com
token: ${VAULT_TOKEN}
```
## Security Headers
```java
http
.headers(headers -> headers
.contentSecurityPolicy(csp -> csp
.policyDirectives("default-src 'self'"))
.frameOptions(HeadersConfigurer.FrameOptionsConfig::sameOrigin)
.xssProtection(Customizer.withDefaults())
.referrerPolicy(rp -> rp.policy(ReferrerPolicyHeaderWriter.ReferrerPolicy.NO_REFERRER)));
```
## CORS Configuration
- Configure CORS at the security filter level, not per-controller
- Restrict allowed origins — never use `*` in production
```java
@Bean
public CorsConfigurationSource corsConfigurationSource() {
CorsConfiguration config = new CorsConfiguration();
config.setAllowedOrigins(List.of("https://app.example.com"));
config.setAllowedMethods(List.of("GET", "POST", "PUT", "DELETE"));
config.setAllowedHeaders(List.of("Authorization", "Content-Type"));
config.setAllowCredentials(true);
config.setMaxAge(3600L);
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
source.registerCorsConfiguration("/api/**", config);
return source;
}
// In SecurityFilterChain:
http.cors(cors -> cors.configurationSource(corsConfigurationSource()));
```
## Rate Limiting
- Apply Bucket4j or gateway-level limits on expensive endpoints
- Log and alert on bursts; return 429 with retry hints
```java
// Using Bucket4j for per-endpoint rate limiting
@Component
public class RateLimitFilter extends OncePerRequestFilter {
private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();
private Bucket createBucket() {
return Bucket.builder()
.addLimit(Bandwidth.classic(100, Refill.intervally(100, Duration.ofMinutes(1))))
.build();
}
@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response,
FilterChain chain) throws ServletException, IOException {
String clientIp = request.getRemoteAddr();
Bucket bucket = buckets.computeIfAbsent(clientIp, k -> createBucket());
if (bucket.tryConsume(1)) {
chain.doFilter(request, response);
} else {
response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value());
response.getWriter().write("{\"error\": \"Rate limit exceeded\"}");
}
}
}
```
## Dependency Security
- Run OWASP Dependency Check / Snyk in CI
- Keep Spring Boot and Spring Security on supported versions
- Fail builds on known CVEs
## Logging and PII
- Never log secrets, tokens, passwords, or full PAN data
- Redact sensitive fields; use structured JSON logging
## File Uploads
- Validate size, content type, and extension
- Store outside web root; scan if required
## Checklist Before Release
- [ ] Auth tokens validated and expired correctly
- [ ] Authorization guards on every sensitive path
- [ ] All inputs validated and sanitized
- [ ] No string-concatenated SQL
- [ ] CSRF posture correct for app type
- [ ] Secrets externalized; none committed
- [ ] Security headers configured
- [ ] Rate limiting on APIs
- [ ] Dependencies scanned and up to date
- [ ] Logs free of sensitive data
**Remember**: Deny by default, validate inputs, least privilege, and secure-by-configuration first.

157
skills/springboot-tdd/SKILL.md Normal file
View File

@@ -0,0 +1,157 @@
---
name: springboot-tdd
description: Test-driven development for Spring Boot using JUnit 5, Mockito, MockMvc, Testcontainers, and JaCoCo. Use when adding features, fixing bugs, or refactoring.
---
# Spring Boot TDD Workflow
TDD guidance for Spring Boot services with 80%+ coverage (unit + integration).
## When to Use
- New features or endpoints
- Bug fixes or refactors
- Adding data access logic or security rules
## Workflow
1) Write tests first (they should fail)
2) Implement minimal code to pass
3) Refactor with tests green
4) Enforce coverage (JaCoCo)
## Unit Tests (JUnit 5 + Mockito)
```java
@ExtendWith(MockitoExtension.class)
class MarketServiceTest {
@Mock MarketRepository repo;
@InjectMocks MarketService service;
@Test
void createsMarket() {
CreateMarketRequest req = new CreateMarketRequest("name", "desc", Instant.now(), List.of("cat"));
when(repo.save(any())).thenAnswer(inv -> inv.getArgument(0));
Market result = service.create(req);
assertThat(result.name()).isEqualTo("name");
verify(repo).save(any());
}
}
```
Patterns:
- Arrange-Act-Assert
- Avoid partial mocks; prefer explicit stubbing
- Use `@ParameterizedTest` for variants
## Web Layer Tests (MockMvc)
```java
@WebMvcTest(MarketController.class)
class MarketControllerTest {
@Autowired MockMvc mockMvc;
@MockBean MarketService marketService;
@Test
void returnsMarkets() throws Exception {
when(marketService.list(any())).thenReturn(Page.empty());
mockMvc.perform(get("/api/markets"))
.andExpect(status().isOk())
.andExpect(jsonPath("$.content").isArray());
}
}
```
## Integration Tests (SpringBootTest)
```java
@SpringBootTest
@AutoConfigureMockMvc
@ActiveProfiles("test")
class MarketIntegrationTest {
@Autowired MockMvc mockMvc;
@Test
void createsMarket() throws Exception {
mockMvc.perform(post("/api/markets")
.contentType(MediaType.APPLICATION_JSON)
.content("""
{"name":"Test","description":"Desc","endDate":"2030-01-01T00:00:00Z","categories":["general"]}
"""))
.andExpect(status().isCreated());
}
}
```
## Persistence Tests (DataJpaTest)
```java
@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Import(TestContainersConfig.class)
class MarketRepositoryTest {
@Autowired MarketRepository repo;
@Test
void savesAndFinds() {
MarketEntity entity = new MarketEntity();
entity.setName("Test");
repo.save(entity);
Optional<MarketEntity> found = repo.findByName("Test");
assertThat(found).isPresent();
}
}
```
## Testcontainers
- Use reusable containers for Postgres/Redis to mirror production
- Wire via `@DynamicPropertySource` to inject JDBC URLs into Spring context
## Coverage (JaCoCo)
Maven snippet:
```xml
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.14</version>
<executions>
<execution>
<goals><goal>prepare-agent</goal></goals>
</execution>
<execution>
<id>report</id>
<phase>verify</phase>
<goals><goal>report</goal></goals>
</execution>
</executions>
</plugin>
```
## Assertions
- Prefer AssertJ (`assertThat`) for readability
- For JSON responses, use `jsonPath`
- For exceptions: `assertThatThrownBy(...)`
## Test Data Builders
```java
class MarketBuilder {
private String name = "Test";
MarketBuilder withName(String name) { this.name = name; return this; }
Market build() { return new Market(null, name, MarketStatus.ACTIVE); }
}
```
## CI Commands
- Maven: `mvn -T 4 test` or `mvn verify`
- Gradle: `./gradlew test jacocoTestReport`
**Remember**: Keep tests fast, isolated, and deterministic. Test behavior, not implementation details.

230
skills/springboot-verification/SKILL.md Normal file
View File

@@ -0,0 +1,230 @@
---
name: springboot-verification
description: "Verification loop for Spring Boot projects: build, static analysis, tests with coverage, security scans, and diff review before release or PR."
---
# Spring Boot Verification Loop
Run before PRs, after major changes, and pre-deploy.
## When to Activate
- Before opening a pull request for a Spring Boot service
- After major refactoring or dependency upgrades
- Pre-deployment verification for staging or production
- Running full build → lint → test → security scan pipeline
- Validating test coverage meets thresholds
## Phase 1: Build
```bash
mvn -T 4 clean verify -DskipTests
# or
./gradlew clean assemble -x test
```
If build fails, stop and fix.
## Phase 2: Static Analysis
Maven (common plugins):
```bash
mvn -T 4 spotbugs:check pmd:check checkstyle:check
```
Gradle (if configured):
```bash
./gradlew checkstyleMain pmdMain spotbugsMain
```
## Phase 3: Tests + Coverage
```bash
mvn -T 4 test
mvn jacoco:report # verify 80%+ coverage
# or
./gradlew test jacocoTestReport
```
Report:
- Total tests, passed/failed
- Coverage % (lines/branches)
### Unit Tests
Test service logic in isolation with mocked dependencies:
```java
@ExtendWith(MockitoExtension.class)
class UserServiceTest {
@Mock private UserRepository userRepository;
@InjectMocks private UserService userService;
@Test
void createUser_validInput_returnsUser() {
var dto = new CreateUserDto("Alice", "alice@example.com");
var expected = new User(1L, "Alice", "alice@example.com");
when(userRepository.save(any(User.class))).thenReturn(expected);
var result = userService.create(dto);
assertThat(result.name()).isEqualTo("Alice");
verify(userRepository).save(any(User.class));
}
@Test
void createUser_duplicateEmail_throwsException() {
var dto = new CreateUserDto("Alice", "existing@example.com");
when(userRepository.existsByEmail(dto.email())).thenReturn(true);
assertThatThrownBy(() -> userService.create(dto))
.isInstanceOf(DuplicateEmailException.class);
}
}
```
### Integration Tests with Testcontainers
Test against a real database instead of H2:
```java
@SpringBootTest
@Testcontainers
class UserRepositoryIntegrationTest {
@Container
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16-alpine")
.withDatabaseName("testdb");
@DynamicPropertySource
static void configureProperties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", postgres::getJdbcUrl);
registry.add("spring.datasource.username", postgres::getUsername);
registry.add("spring.datasource.password", postgres::getPassword);
}
@Autowired private UserRepository userRepository;
@Test
void findByEmail_existingUser_returnsUser() {
userRepository.save(new User("Alice", "alice@example.com"));
var found = userRepository.findByEmail("alice@example.com");
assertThat(found).isPresent();
assertThat(found.get().getName()).isEqualTo("Alice");
}
}
```
### API Tests with MockMvc
Test controller layer with full Spring context:
```java
@WebMvcTest(UserController.class)
class UserControllerTest {
@Autowired private MockMvc mockMvc;
@MockBean private UserService userService;
@Test
void createUser_validInput_returns201() throws Exception {
var user = new UserDto(1L, "Alice", "alice@example.com");
when(userService.create(any())).thenReturn(user);
mockMvc.perform(post("/api/users")
.contentType(MediaType.APPLICATION_JSON)
.content("""
{"name": "Alice", "email": "alice@example.com"}
"""))
.andExpect(status().isCreated())
.andExpect(jsonPath("$.name").value("Alice"));
}
@Test
void createUser_invalidEmail_returns400() throws Exception {
mockMvc.perform(post("/api/users")
.contentType(MediaType.APPLICATION_JSON)
.content("""
{"name": "Alice", "email": "not-an-email"}
"""))
.andExpect(status().isBadRequest());
}
}
```
## Phase 4: Security Scan
```bash
# Dependency CVEs
mvn org.owasp:dependency-check-maven:check
# or
./gradlew dependencyCheckAnalyze
# Secrets in source
grep -rn "password\s*=\s*\"" src/ --include="*.java" --include="*.yml" --include="*.properties"
grep -rn "sk-\|api_key\|secret" src/ --include="*.java" --include="*.yml"
# Secrets (git history)
git secrets --scan # if configured
```
### Common Security Findings
```
# Check for System.out.println (use logger instead)
grep -rn "System\.out\.print" src/main/ --include="*.java"
# Check for raw exception messages in responses
grep -rn "e\.getMessage()" src/main/ --include="*.java"
# Check for wildcard CORS
grep -rn "allowedOrigins.*\*" src/main/ --include="*.java"
```
## Phase 5: Lint/Format (optional gate)
```bash
mvn spotless:apply # if using Spotless plugin
./gradlew spotlessApply
```
## Phase 6: Diff Review
```bash
git diff --stat
git diff
```
Checklist:
- No debugging logs left (`System.out`, `log.debug` without guards)
- Meaningful errors and HTTP statuses
- Transactions and validation present where needed
- Config changes documented
## Output Template
```
VERIFICATION REPORT
===================
Build: [PASS/FAIL]
Static: [PASS/FAIL] (spotbugs/pmd/checkstyle)
Tests: [PASS/FAIL] (X/Y passed, Z% coverage)
Security: [PASS/FAIL] (CVE findings: N)
Diff: [X files changed]
Overall: [READY / NOT READY]
Issues to Fix:
1. ...
2. ...
```
## Continuous Mode
- Re-run phases on significant changes or every 30–60 minutes in long sessions
- Keep a short loop: `mvn -T 4 test` + spotbugs for quick feedback
**Remember**: Fast feedback beats late surprises. Keep the gate strict—treat warnings as defects in production systems.

102
skills/strategic-compact/SKILL.md Normal file
View File

@@ -0,0 +1,102 @@
---
name: strategic-compact
description: Suggests manual context compaction at logical intervals to preserve context through task phases rather than arbitrary auto-compaction.
---
# Strategic Compact Skill
Suggests manual `/compact` at strategic points in your workflow rather than relying on arbitrary auto-compaction.
## When to Activate
- Running long sessions that approach context limits (200K+ tokens)
- Working on multi-phase tasks (research → plan → implement → test)
- Switching between unrelated tasks within the same session
- After completing a major milestone and starting new work
- When responses slow down or become less coherent (context pressure)
## Why Strategic Compaction?
Auto-compaction triggers at arbitrary points:
- Often mid-task, losing important context
- No awareness of logical task boundaries
- Can interrupt complex multi-step operations
Strategic compaction at logical boundaries:
- **After exploration, before execution** — Compact research context, keep implementation plan
- **After completing a milestone** — Fresh start for next phase
- **Before major context shifts** — Clear exploration context before different task
## How It Works
The `suggest-compact.js` script runs on PreToolUse (Edit/Write) and:
1. **Tracks tool calls** — Counts tool invocations in session
2. **Threshold detection** — Suggests at configurable threshold (default: 50 calls)
3. **Periodic reminders** — Reminds every 25 calls after threshold
## Hook Setup
Add to your `~/.claude/settings.json`:
```json
{
"hooks": {
"PreToolUse": [
{
"matcher": "Edit",
"hooks": [{ "type": "command", "command": "node ~/.claude/skills/strategic-compact/suggest-compact.js" }]
},
{
"matcher": "Write",
"hooks": [{ "type": "command", "command": "node ~/.claude/skills/strategic-compact/suggest-compact.js" }]
}
]
}
}
```
## Configuration
Environment variables:
- `COMPACT_THRESHOLD` — Tool calls before first suggestion (default: 50)
## Compaction Decision Guide
Use this table to decide when to compact:
| Phase Transition | Compact? | Why |
|-----------------|----------|-----|
| Research → Planning | Yes | Research context is bulky; plan is the distilled output |
| Planning → Implementation | Yes | Plan is in TodoWrite or a file; free up context for code |
| Implementation → Testing | Maybe | Keep if tests reference recent code; compact if switching focus |
| Debugging → Next feature | Yes | Debug traces pollute context for unrelated work |
| Mid-implementation | No | Losing variable names, file paths, and partial state is costly |
| After a failed approach | Yes | Clear the dead-end reasoning before trying a new approach |
## What Survives Compaction
Understanding what persists helps you compact with confidence:
| Persists | Lost |
|----------|------|
| CLAUDE.md instructions | Intermediate reasoning and analysis |
| TodoWrite task list | File contents you previously read |
| Memory files (`~/.claude/memory/`) | Multi-step conversation context |
| Git state (commits, branches) | Tool call history and counts |
| Files on disk | Nuanced user preferences stated verbally |
## Best Practices
1. **Compact after planning** — Once plan is finalized in TodoWrite, compact to start fresh
2. **Compact after debugging** — Clear error-resolution context before continuing
3. **Don't compact mid-implementation** — Preserve context for related changes
4. **Read the suggestion** — The hook tells you *when*, you decide *if*
5. **Write before compacting** — Save important context to files or memory before compacting
6. **Use `/compact` with a summary** — Add a custom message: `/compact Focus on implementing auth middleware next`
## Related
- [The Longform Guide](https://x.com/affaanmustafa/status/2014040193557471352) — Token optimization section
- Memory persistence hooks — For state that survives compaction
- `continuous-learning` skill — Extracts patterns before session ends

54
skills/strategic-compact/suggest-compact.sh Executable file
View File

@@ -0,0 +1,54 @@
#!/bin/bash
# Strategic Compact Suggester
# Runs on PreToolUse or periodically to suggest manual compaction at logical intervals
#
# Why manual over auto-compact:
# - Auto-compact happens at arbitrary points, often mid-task
# - Strategic compacting preserves context through logical phases
# - Compact after exploration, before execution
# - Compact after completing a milestone, before starting next
#
# Hook config (in ~/.claude/settings.json):
# {
# "hooks": {
# "PreToolUse": [{
# "matcher": "Edit|Write",
# "hooks": [{
# "type": "command",
# "command": "~/.claude/skills/strategic-compact/suggest-compact.sh"
# }]
# }]
# }
# }
#
# Criteria for suggesting compact:
# - Session has been running for extended period
# - Large number of tool calls made
# - Transitioning from research/exploration to implementation
# - Plan has been finalized
# Track tool call count (increment in a temp file)
# Use CLAUDE_SESSION_ID for session-specific counter (not $$ which changes per invocation)
SESSION_ID="${CLAUDE_SESSION_ID:-${PPID:-default}}"
COUNTER_FILE="/tmp/claude-tool-count-${SESSION_ID}"
THRESHOLD=${COMPACT_THRESHOLD:-50}
# Initialize or increment counter
if [ -f "$COUNTER_FILE" ]; then
count=$(cat "$COUNTER_FILE")
count=$((count + 1))
echo "$count" > "$COUNTER_FILE"
else
echo "1" > "$COUNTER_FILE"
count=1
fi
# Suggest compact after threshold tool calls
if [ "$count" -eq "$THRESHOLD" ]; then
echo "[StrategicCompact] $THRESHOLD tool calls reached - consider /compact if transitioning phases" >&2
fi
# Suggest at regular intervals after threshold
if [ "$count" -gt "$THRESHOLD" ] && [ $((count % 25)) -eq 0 ]; then
echo "[StrategicCompact] $count tool calls - good checkpoint for /compact if context is stale" >&2
fi

59
skills/sysadmin.md Normal file
View File

@@ -0,0 +1,59 @@
# System Administration Skill
## System Info
```bash
uname -a # Kernel info
cat /etc/os-release # OS details
hostname # Hostname
uptime # System uptime
```
## Process Management
```bash
ps aux # All processes
top # Interactive view
htop # Better top
kill -9 pid # Force kill
pkill process_name # Kill by name
```
## Memory & Disk
```bash
free -h # Memory usage
df -h # Disk usage
du -sh folder # Folder size
```
## Network
```bash
ip addr # IP addresses
netstat -tulpn # Listening ports
ss -tulpn # Alternative to netstat
curl url # HTTP request
wget url # Download
```
## Services
```bash
systemctl status service
systemctl start service
systemctl stop service
systemctl restart service
systemctl enable service
```
## User Management
```bash
whoami # Current user
id # User info
useradd name # Add user
usermod -aG group user # Add to group
passwd user # Change password
```
## Logs
```bash
journalctl -u service # Service logs
tail -f /var/log/syslog
dmesg | tail
```

409
skills/tdd-workflow/SKILL.md Normal file
View File

@@ -0,0 +1,409 @@
---
name: tdd-workflow
description: Use this skill when writing new features, fixing bugs, or refactoring code. Enforces test-driven development with 80%+ coverage including unit, integration, and E2E tests.
---
# Test-Driven Development Workflow
This skill ensures all code development follows TDD principles with comprehensive test coverage.
## When to Activate
- Writing new features or functionality
- Fixing bugs or issues
- Refactoring existing code
- Adding API endpoints
- Creating new components
## Core Principles
### 1. Tests BEFORE Code
ALWAYS write tests first, then implement code to make tests pass.
### 2. Coverage Requirements
- Minimum 80% coverage (unit + integration + E2E)
- All edge cases covered
- Error scenarios tested
- Boundary conditions verified
### 3. Test Types
#### Unit Tests
- Individual functions and utilities
- Component logic
- Pure functions
- Helpers and utilities
#### Integration Tests
- API endpoints
- Database operations
- Service interactions
- External API calls
#### E2E Tests (Playwright)
- Critical user flows
- Complete workflows
- Browser automation
- UI interactions
## TDD Workflow Steps
### Step 1: Write User Journeys
```
As a [role], I want to [action], so that [benefit]
Example:
As a user, I want to search for markets semantically,
so that I can find relevant markets even without exact keywords.
```
### Step 2: Generate Test Cases
For each user journey, create comprehensive test cases:
```typescript
describe('Semantic Search', () => {
it('returns relevant markets for query', async () => {
// Test implementation
})
it('handles empty query gracefully', async () => {
// Test edge case
})
it('falls back to substring search when Redis unavailable', async () => {
// Test fallback behavior
})
it('sorts results by similarity score', async () => {
// Test sorting logic
})
})
```
### Step 3: Run Tests (They Should Fail)
```bash
npm test
# Tests should fail - we haven't implemented yet
```
### Step 4: Implement Code
Write minimal code to make tests pass:
```typescript
// Implementation guided by tests
export async function searchMarkets(query: string) {
// Implementation here
}
```
### Step 5: Run Tests Again
```bash
npm test
# Tests should now pass
```
### Step 6: Refactor
Improve code quality while keeping tests green:
- Remove duplication
- Improve naming
- Optimize performance
- Enhance readability
### Step 7: Verify Coverage
```bash
npm run test:coverage
# Verify 80%+ coverage achieved
```
## Testing Patterns
### Unit Test Pattern (Jest/Vitest)
```typescript
import { render, screen, fireEvent } from '@testing-library/react'
import { Button } from './Button'
describe('Button Component', () => {
it('renders with correct text', () => {
render(<Button>Click me</Button>)
expect(screen.getByText('Click me')).toBeInTheDocument()
})
it('calls onClick when clicked', () => {
const handleClick = jest.fn()
render(<Button onClick={handleClick}>Click</Button>)
fireEvent.click(screen.getByRole('button'))
expect(handleClick).toHaveBeenCalledTimes(1)
})
it('is disabled when disabled prop is true', () => {
render(<Button disabled>Click</Button>)
expect(screen.getByRole('button')).toBeDisabled()
})
})
```
### API Integration Test Pattern
```typescript
import { NextRequest } from 'next/server'
import { GET } from './route'
describe('GET /api/markets', () => {
it('returns markets successfully', async () => {
const request = new NextRequest('http://localhost/api/markets')
const response = await GET(request)
const data = await response.json()
expect(response.status).toBe(200)
expect(data.success).toBe(true)
expect(Array.isArray(data.data)).toBe(true)
})
it('validates query parameters', async () => {
const request = new NextRequest('http://localhost/api/markets?limit=invalid')
const response = await GET(request)
expect(response.status).toBe(400)
})
it('handles database errors gracefully', async () => {
// Mock database failure
const request = new NextRequest('http://localhost/api/markets')
// Test error handling
})
})
```
### E2E Test Pattern (Playwright)
```typescript
import { test, expect } from '@playwright/test'
test('user can search and filter markets', async ({ page }) => {
// Navigate to markets page
await page.goto('/')
await page.click('a[href="/markets"]')
// Verify page loaded
await expect(page.locator('h1')).toContainText('Markets')
// Search for markets
await page.fill('input[placeholder="Search markets"]', 'election')
// Wait for debounce and results
await page.waitForTimeout(600)
// Verify search results displayed
const results = page.locator('[data-testid="market-card"]')
await expect(results).toHaveCount(5, { timeout: 5000 })
// Verify results contain search term
const firstResult = results.first()
await expect(firstResult).toContainText('election', { ignoreCase: true })
// Filter by status
await page.click('button:has-text("Active")')
// Verify filtered results
await expect(results).toHaveCount(3)
})
test('user can create a new market', async ({ page }) => {
// Login first
await page.goto('/creator-dashboard')
// Fill market creation form
await page.fill('input[name="name"]', 'Test Market')
await page.fill('textarea[name="description"]', 'Test description')
await page.fill('input[name="endDate"]', '2025-12-31')
// Submit form
await page.click('button[type="submit"]')
// Verify success message
await expect(page.locator('text=Market created successfully')).toBeVisible()
// Verify redirect to market page
await expect(page).toHaveURL(/\/markets\/test-market/)
})
```
## Test File Organization
```
src/
├── components/
│ ├── Button/
│ │ ├── Button.tsx
│ │ ├── Button.test.tsx # Unit tests
│ │ └── Button.stories.tsx # Storybook
│ └── MarketCard/
│ ├── MarketCard.tsx
│ └── MarketCard.test.tsx
├── app/
│ └── api/
│ └── markets/
│ ├── route.ts
│ └── route.test.ts # Integration tests
└── e2e/
├── markets.spec.ts # E2E tests
├── trading.spec.ts
└── auth.spec.ts
```
## Mocking External Services
### Supabase Mock
```typescript
jest.mock('@/lib/supabase', () => ({
supabase: {
from: jest.fn(() => ({
select: jest.fn(() => ({
eq: jest.fn(() => Promise.resolve({
data: [{ id: 1, name: 'Test Market' }],
error: null
}))
}))
}))
}
}))
```
### Redis Mock
```typescript
jest.mock('@/lib/redis', () => ({
searchMarketsByVector: jest.fn(() => Promise.resolve([
{ slug: 'test-market', similarity_score: 0.95 }
])),
checkRedisHealth: jest.fn(() => Promise.resolve({ connected: true }))
}))
```
### OpenAI Mock
```typescript
jest.mock('@/lib/openai', () => ({
generateEmbedding: jest.fn(() => Promise.resolve(
new Array(1536).fill(0.1) // Mock 1536-dim embedding
))
}))
```
## Test Coverage Verification
### Run Coverage Report
```bash
npm run test:coverage
```
### Coverage Thresholds
```json
{
"jest": {
"coverageThresholds": {
"global": {
"branches": 80,
"functions": 80,
"lines": 80,
"statements": 80
}
}
}
}
```
## Common Testing Mistakes to Avoid
### ❌ WRONG: Testing Implementation Details
```typescript
// Don't test internal state
expect(component.state.count).toBe(5)
```
### ✅ CORRECT: Test User-Visible Behavior
```typescript
// Test what users see
expect(screen.getByText('Count: 5')).toBeInTheDocument()
```
### ❌ WRONG: Brittle Selectors
```typescript
// Breaks easily
await page.click('.css-class-xyz')
```
### ✅ CORRECT: Semantic Selectors
```typescript
// Resilient to changes
await page.click('button:has-text("Submit")')
await page.click('[data-testid="submit-button"]')
```
### ❌ WRONG: No Test Isolation
```typescript
// Tests depend on each other
test('creates user', () => { /* ... */ })
test('updates same user', () => { /* depends on previous test */ })
```
### ✅ CORRECT: Independent Tests
```typescript
// Each test sets up its own data
test('creates user', () => {
const user = createTestUser()
// Test logic
})
test('updates user', () => {
const user = createTestUser()
// Update logic
})
```
## Continuous Testing
### Watch Mode During Development
```bash
npm test -- --watch
# Tests run automatically on file changes
```
### Pre-Commit Hook
```bash
# Runs before every commit
npm test && npm run lint
```
### CI/CD Integration
```yaml
# GitHub Actions
- name: Run Tests
run: npm test -- --coverage
- name: Upload Coverage
uses: codecov/codecov-action@v3
```
## Best Practices
1. **Write Tests First** - Always TDD
2. **One Assert Per Test** - Focus on single behavior
3. **Descriptive Test Names** - Explain what's tested
4. **Arrange-Act-Assert** - Clear test structure
5. **Mock External Dependencies** - Isolate unit tests
6. **Test Edge Cases** - Null, undefined, empty, large
7. **Test Error Paths** - Not just happy paths
8. **Keep Tests Fast** - Unit tests < 50ms each
9. **Clean Up After Tests** - No side effects
10. **Review Coverage Reports** - Identify gaps
## Success Metrics
- 80%+ code coverage achieved
- All tests passing (green)
- No skipped or disabled tests
- Fast test execution (< 30s for unit tests)
- E2E tests cover critical user flows
- Tests catch bugs before production
---
**Remember**: Tests are not optional. They are the safety net that enables confident refactoring, rapid development, and production reliability.

125
skills/verification-loop/SKILL.md Normal file
View File

@@ -0,0 +1,125 @@
---
name: verification-loop
description: "A comprehensive verification system for Claude Code sessions."
---
# Verification Loop Skill
A comprehensive verification system for Claude Code sessions.
## When to Use
Invoke this skill:
- After completing a feature or significant code change
- Before creating a PR
- When you want to ensure quality gates pass
- After refactoring
## Verification Phases
### Phase 1: Build Verification
```bash
# Check if project builds
npm run build 2>&1 | tail -20
# OR
pnpm build 2>&1 | tail -20
```
If build fails, STOP and fix before continuing.
### Phase 2: Type Check
```bash
# TypeScript projects
npx tsc --noEmit 2>&1 | head -30
# Python projects
pyright . 2>&1 | head -30
```
Report all type errors. Fix critical ones before continuing.
### Phase 3: Lint Check
```bash
# JavaScript/TypeScript
npm run lint 2>&1 | head -30
# Python
ruff check . 2>&1 | head -30
```
### Phase 4: Test Suite
```bash
# Run tests with coverage
npm run test -- --coverage 2>&1 | tail -50
# Check coverage threshold
# Target: 80% minimum
```
Report:
- Total tests: X
- Passed: X
- Failed: X
- Coverage: X%
### Phase 5: Security Scan
```bash
# Check for secrets
grep -rn "sk-" --include="*.ts" --include="*.js" . 2>/dev/null | head -10
grep -rn "api_key" --include="*.ts" --include="*.js" . 2>/dev/null | head -10
# Check for console.log
grep -rn "console.log" --include="*.ts" --include="*.tsx" src/ 2>/dev/null | head -10
```
### Phase 6: Diff Review
```bash
# Show what changed
git diff --stat
git diff HEAD~1 --name-only
```
Review each changed file for:
- Unintended changes
- Missing error handling
- Potential edge cases
## Output Format
After running all phases, produce a verification report:
```
VERIFICATION REPORT
==================
Build: [PASS/FAIL]
Types: [PASS/FAIL] (X errors)
Lint: [PASS/FAIL] (X warnings)
Tests: [PASS/FAIL] (X/Y passed, Z% coverage)
Security: [PASS/FAIL] (X issues)
Diff: [X files changed]
Overall: [READY/NOT READY] for PR
Issues to Fix:
1. ...
2. ...
```
## Continuous Mode
For long sessions, run verification every 15 minutes or after major changes:
```markdown
Set a mental checkpoint:
- After completing each function
- After finishing a component
- Before moving to next task
Run: /verify
```
## Integration with Hooks
This skill complements PostToolUse hooks but provides deeper verification.
Hooks catch issues immediately; this skill provides comprehensive review.