A production-grade backend service for a real-time collaborative workspace platform, similar to collaborative coding environments. Built with scalability, security, and performance in mind.
- Features
- Tech Stack
- Architecture
- Prerequisites
- Installation
- Configuration
- Running the Application
- API Documentation
- Testing
- Deployment
- Security
- Performance
- Design Decisions
- JWT-based authentication with access and refresh tokens
- Role-based access control (Owner, Collaborator, Viewer)
- Token refresh mechanism
- API rate limiting (general, auth-specific, and API-specific)
- Secure password hashing with bcrypt
- RESTful APIs for CRUD operations
- Project ownership and status management
- Workspace creation and management
- Collaborator invitation and role assignment
- OpenAPI/Swagger documentation
- WebSocket-based communication using Socket.io
- Event broadcasting (user join/leave, file changes, cursor movements, activities)
- Redis Pub/Sub for horizontal scaling
- Activity tracking and persistence
- Bull queue (Redis-based) for job management
- Multiple job types (code execution, file processing, workspace export)
- Retry logic with exponential backoff
- Failure handling and error tracking
- Idempotent job processing
- Job status tracking and monitoring
- PostgreSQL for relational data (users, projects, workspaces, collaborators)
- MongoDB for document storage (jobs, activities)
- Redis for caching and message brokering
- Proper indexing and query optimization
- Redis caching with TTL
- Horizontally scalable architecture
- Async/non-blocking I/O
- Comprehensive unit and integration tests (>70% coverage)
- Docker and Docker Compose setup
- CI/CD pipeline with GitHub Actions
- Input validation and sanitization
- SQL/NoSQL injection protection
- Environment-based configuration
- CORS configuration
- Structured logging with Winston
- Health checks and monitoring
- Feature flags
- Clean architecture patterns
- Runtime: Node.js 18+
- Language: TypeScript
- Framework: Express.js
- PostgreSQL: Relational data storage
- MongoDB: Document storage for logs and activities
- Redis: Caching and message broker
- Socket.io: WebSocket communication
- Bull: Job queue management
- Redis Pub/Sub: Event distribution
- Jest: Testing framework
- Supertest: HTTP assertions
- ts-jest: TypeScript support for Jest
- Docker: Containerization
- Docker Compose: Multi-container orchestration
- GitHub Actions: CI/CD pipeline
- Helmet: Security headers
- express-rate-limit: Rate limiting
- express-validator: Request validation
- jsonwebtoken: JWT implementation
- bcryptjs: Password hashing
┌─────────────────────────────────────────────────────────────────┐
│ Load Balancer │
└───────────────────────────┬─────────────────────────────────────┘
│
┌───────────────┴───────────────┐
│ │
┌───────────▼──────────┐ ┌───────────▼──────────┐
│ API Server (1) │ │ API Server (2) │
│ - REST APIs │ │ - REST APIs │
│ - WebSocket │ │ - WebSocket │
└───────┬──────────────┘ └───────┬──────────────┘
│ │
└───────────────┬───────────────┘
│
┌───────────────┼───────────────┐
│ │ │
┌───────▼──────┐ ┌──────▼─────┐ ┌──────▼─────┐
│ PostgreSQL │ │ MongoDB │ │ Redis │
│ (Relational)│ │ (Documents)│ │ (Cache & │
│ │ │ │ │ Queue) │
└──────────────┘ └────────────┘ └──────┬─────┘
│
┌──────▼─────┐
│ Worker │
│ (Bull) │
└────────────┘
- Layered Architecture: Controllers → Services → Database
- Repository Pattern: Database abstraction with TypeORM and Mongoose
- Dependency Injection: Configuration-based service initialization
- Middleware Pattern: Authentication, validation, error handling
- Pub/Sub Pattern: Redis-based event distribution for scalability
- Queue Pattern: Asynchronous job processing with Bull
- Node.js 18.x or higher
- npm 9.x or higher
- Docker and Docker Compose (for containerized deployment)
- PostgreSQL 15+ (if running locally)
- MongoDB 7+ (if running locally)
- Redis 7+ (if running locally)
git clone https://github.com/yourusername/collaborative-workspace.git
cd collaborative-workspacenpm installcp .env.example .envEdit .env with your configuration values.
Key environment variables:
# Application
NODE_ENV=development
PORT=3000
API_VERSION=v1
# JWT
JWT_SECRET=your-secret-key
JWT_REFRESH_SECRET=your-refresh-secret
JWT_EXPIRES_IN=15m
JWT_REFRESH_EXPIRES_IN=7d
# PostgreSQL
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_USER=workspace_user
POSTGRES_PASSWORD=workspace_password
POSTGRES_DB=collaborative_workspace
# MongoDB
MONGODB_URI=mongodb://localhost:27017/collaborative_workspace
# Redis
REDISHOST=localhost
REDISPORT=6379
# Rate Limiting
RATE_LIMIT_WINDOW_MS=900000
RATE_LIMIT_MAX_REQUESTS=100
# Feature Flags
FEATURE_ADVANCED_ANALYTICS=false
FEATURE_CODE_EXECUTION=true# Start all services with Docker Compose
docker-compose up -d
# Or run locally (requires databases running)
npm run dev
# Run worker in separate terminal
npm run worker# Build TypeScript
npm run build
# Start production server
npm start# Build and start all services
docker-compose up -d
# View logs
docker-compose logs -f
# Stop all services
docker-compose downOnce the server is running, access the interactive Swagger documentation at:
http://localhost:3000/api-docs
POST /api/v1/auth/register- Register new userPOST /api/v1/auth/login- User loginPOST /api/v1/auth/refresh- Refresh access tokenPOST /api/v1/auth/logout- User logout
POST /api/v1/projects- Create projectGET /api/v1/projects- List projectsGET /api/v1/projects/:id- Get project detailsPUT /api/v1/projects/:id- Update projectDELETE /api/v1/projects/:id- Delete project
POST /api/v1/workspaces- Create workspaceGET /api/v1/workspaces/project/:projectId- List workspacesGET /api/v1/workspaces/:id- Get workspace detailsPUT /api/v1/workspaces/:id- Update workspaceDELETE /api/v1/workspaces/:id- Delete workspacePOST /api/v1/workspaces/:id/collaborators- Invite collaboratorPUT /api/v1/workspaces/:id/collaborators/:collaboratorId- Update roleDELETE /api/v1/workspaces/:id/collaborators/:collaboratorId- Remove collaborator
POST /api/v1/jobs- Create jobGET /api/v1/jobs- List user jobsGET /api/v1/jobs/:id- Get job statusPOST /api/v1/jobs/:id/retry- Retry failed job
Client → Server:
join_workspace- Join a workspace roomleave_workspace- Leave a workspace roomfile_change- Broadcast file changescursor_move- Broadcast cursor positionactivity_update- Broadcast activity updates
Server → Client:
user_joined- User joined workspaceuser_left- User left workspacefile_change- File was changedcursor_move- Cursor movedactivity_update- Activity updated
# Register
curl -X POST http://localhost:3000/api/v1/auth/register \
-H "Content-Type: application/json" \
-d '{
"email": "user@example.com",
"password": "SecurePass123",
"firstName": "John",
"lastName": "Doe"
}'
# Login
curl -X POST http://localhost:3000/api/v1/auth/login \
-H "Content-Type: application/json" \
-d '{
"email": "user@example.com",
"password": "SecurePass123"
}'curl -X POST http://localhost:3000/api/v1/projects \
-H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "My Project",
"description": "Project description"
}'import io from 'socket.io-client';
const socket = io('http://localhost:3000', {
auth: {
token: 'YOUR_ACCESS_TOKEN'
}
});
socket.on('connect', () => {
console.log('Connected');
socket.emit('join_workspace', 'workspace-id');
});
socket.on('user_joined', (data) => {
console.log('User joined:', data);
});
socket.emit('file_change', {
fileName: 'index.ts',
changes: { /* ... */ }
});npm testnpm test -- --coveragenpm test -- authService.test.tsnpm run test:integrationTests achieve >70% code coverage across:
- Unit tests for services and utilities
- Integration tests for API endpoints
- Cache service tests
- Authentication flow tests
# Production deployment
docker-compose -f docker-compose.yml up -d
# Check status
docker-compose ps
# View logs
docker-compose logs -f apiCreate Kubernetes manifests based on the Docker setup:
- Deployment for API servers
- Deployment for workers
- StatefulSet for databases
- Services for load balancing
- ConfigMaps for configuration
- Secrets for sensitive data
AWS:
- ECS/EKS for container orchestration
- RDS for PostgreSQL
- DocumentDB for MongoDB
- ElastiCache for Redis
- ALB for load balancing
Google Cloud:
- GKE for Kubernetes
- Cloud SQL for PostgreSQL
- MongoDB Atlas
- Memorystore for Redis
Azure:
- AKS for Kubernetes
- Azure Database for PostgreSQL
- Cosmos DB
- Azure Cache for Redis
- Authentication: JWT with short-lived access tokens and long-lived refresh tokens
- Authorization: Role-based access control (RBAC)
- Rate Limiting: Multiple rate limiters (general, auth, API-specific)
- Input Validation: express-validator for all inputs
- SQL Injection Protection: TypeORM parameterized queries
- NoSQL Injection Protection: Mongoose schema validation
- Helmet: Security headers (XSS, CSP, etc.)
- CORS: Configurable origin restrictions
- Password Security: bcrypt with salt rounds
- Environment Variables: Secure secrets management
- Error Handling: No sensitive data in error responses
- Always use HTTPS in production
- Rotate JWT secrets regularly
- Keep dependencies updated
- Use secrets management services (AWS Secrets Manager, HashiCorp Vault)
- Enable database encryption at rest
- Implement API gateway for additional security layer
- Monitor and log security events
- Regular security audits
- Caching: Redis caching with TTL for frequently accessed data
- Database Indexing: Proper indexes on frequently queried fields
- Connection Pooling: Database connection management
- Async Processing: Bull queue for heavy operations
- Horizontal Scaling: Stateless API servers with Redis Pub/Sub
- Query Optimization: Efficient database queries with pagination
- Compression: Response compression (gzip)
- Load Balancing: Support for multiple server instances
- Stateless Design: API servers can be scaled horizontally
- Redis Pub/Sub: WebSocket events distributed across instances
- Job Queue: Workers can be scaled independently
- Database Sharding: PostgreSQL and MongoDB support sharding
- CDN Integration: Static assets can be served via CDN
- Microservices Ready: Architecture supports service decomposition
- Type safety reduces runtime errors
- Better IDE support and autocomplete
- Self-documenting code
- Easier refactoring
- PostgreSQL: ACID compliance for critical data (users, projects)
- MongoDB: Flexible schema for logs and activities
- Redis: High-performance caching and messaging
- Automatic reconnection
- Room support for workspace isolation
- Fallback mechanisms
- Better developer experience
- Redis-based (already in stack)
- Excellent TypeScript support
- Built-in retry logic
- Good monitoring capabilities
- Mature and battle-tested
- Large ecosystem
- Flexible middleware system
- Excellent TypeScript support
- Complexity vs Features: Multi-database architecture adds complexity but provides flexibility
- Consistency vs Performance: Eventual consistency for some features (activities) for better performance
- Memory vs Speed: Redis caching uses memory for faster response times
- Development Speed vs Type Safety: TypeScript adds development overhead but prevents bugs
- Structured logging with Winston
- Log levels: error, warn, info, debug
- Separate log files for different levels
- Log rotation recommended for production
- Health check endpoint (
/health) - Job queue metrics (Bull dashboard integration possible)
- Database connection pooling metrics
- API response times
- Application Monitoring: New Relic, Datadog
- Log Aggregation: ELK Stack, Splunk
- Error Tracking: Sentry
- Uptime Monitoring: Pingdom, UptimeRobot



















