atlas commerce

Atlas Commerce — Project Summary

A full-stack, multi-tenant e-commerce marketplace platform (Q-comm / Blinkit-style) with microservices backend, three customer-facing storefronts, an AI-powered website builder, real-time chat, event-driven analytics, and cloud-native DevOps tooling.

Production brand: Penderghast (penderghast.com) Stack at a glance: TypeScript microservices (Express) + Go chat service · Next.js frontends · MongoDB (Prisma) + PostgreSQL · Redis · Apache Kafka · Stripe · ImageKit · TensorFlow.js · Kubernetes / Terraform / Docker.

1. Repository Overview

• Monorepo managed with pnpm workspaces (pnpm@10.4.1) and Turborepo (turbo.json).
• Requires Node >= 20.
• 19 apps under apps/, shared packages under packages/.
• Common workflows: pnpm dev (starts everything with --concurrency=18), pnpm build, pnpm lint, pnpm generate (Prisma), pnpm studio (Prisma Studio).

Directory layout

apps/ 19 applications (15 backend services + 4–5 frontends)
packages/ shared workspace packages
db/ Prisma singleton client + MongoDB schema
error-handler/ typed error classes + Express error middleware
middleware/ JWT auth + role authorization (isAuthenticated, isSeller, isAdmin, isUser)
ui/ @workspace/ui — shadcn/ui design system
eslint-config/ @workspace/eslint-config (base, next-js, react-internal)
typescript-config/ @workspace/typescript-config (base, nextjs, react-library)
libs/
imagekit/ @atlas/imagekit — preconfigured ImageKit SDK client
redis/ @repo/redis — ioredis client + chat unseen-count helpers
utils/
kafka/ @atlas/kafka — shared kafkajs client (env-aware broker resolution)
logs/ @atlas/logs — structured log publisher to Kafka `logs` topic
AI/ learning/sandbox: mini-GPT (PyTorch), RAG, ML examples
devops/ terraform (AWS EKS), kubernetes (kustomize), jenkins, k6 load tests
scripts/ Ghast Chat lifecycle scripts (bash + bat)
generated/prisma/ generated Prisma client (gitignored)
docker-compose.dev.yml dev infra (kafka, postgres, redis, ghast chat stack)
docker-compose.production.yml full platform production compose

2. High-Level Architecture

The platform is a microservice mesh behind an API gateway. All frontends talk to the api-gateway (port 8080), which reverse-proxies by URL prefix to individual services. Services share JWT auth middleware, a Prisma client, and stream analytics via Kafka.

Service / port map

ServicePortRoleapi-gateway8080Reverse proxy, CORS whitelist, rate limiting (1000 req/15 min/IP)auth-service6001Registration/login (user, seller, admin), OTP, JWT, password reset, seller onboarding, Stripe Connect, shipping addresses, Swaggerproduct-service6002Catalog, product CRUD, discount codes, events/offers, search/filters, ImageKit image upload, product analyticsseller-service6003Shop lifecycle (soft-delete + 28-day purge cron), profiles, shop websites + domains, follow/unfollow, notifications, shop members, Swaggerorder-service6004Stripe PaymentIntents/Connect (10% platform fee), webhooks, order lifecycle, delivery status, coupons, confirmation emailsadmin-service6005Admin dashboard backend: products, events, users, sellers, admins, notifications, site configchatting-service6006Buyer↔seller messaging: WebSocket presence, Kafka-persisted messages, unseen countsrecommendation-service6007Personalized recommendations via TensorFlow.js collaborative-filtering model (trained on the fly, 3h cache)logger-service6008Real-time log streaming over WebSocket (Kafka logs topic consumer)kafka-service—Background worker: consumes user-events, updates user/product analyticsoAuth-service4000Minimal OAuth 2.0 authorization-code flow (oauth_clients/codes/tokens)ghast-chat-service8090Go (1.21) chat service: WebSocket rooms, Postgres + Redis, REST APIshopify-app-service3005Next.js — standalone Shopify app simulator ("Atlas Flow Ops"), full OAuth/webhook/theme-extension simulation (in-memory)shipping-service—Empty placeholder directory (no code)web3001Customer storefrontseller-ui3000Seller dashboardadmin-ui3002Admin consoleghast-chat-web3100WhatsApp-style chat client (paired with Go service)website-builder3004AI-agentic drag-and-drop storefront builder

Gateway proxy table

Path prefixTargets/authauth-service :6001/productproduct-service :6002/sellerseller-service :6003/orderorder-service :6004/adminadmin-service :6005/chattingchatting-service :6006/recommendationrecommendation-service :6007/assetsstatic files

3. Frontend Applications

All frontends are Next.js App Router apps sharing the @workspace/ui design system, cookie-based auth via an axios interceptor (401 → refresh-token → retry-queue), and react-query for server state.

web (port 3001) — Customer Storefront

• Stack: Next.js 16.1.6, React 19, Tailwind CSS v4, zustand (persisted cart/wishlist), react-query, sonner, next-themes.
• Pages: Homepage (categories, hero, promo grid, recommended-for-you, fresh arrivals, top offers), /products, /product/[slug] (server component w/ SEO metadata), /shop/[id], /shops, /stores, /cart, /checkout, /payment-success, /order, /wishlist, /offers, /profile, /inbox.
• Event streaming: zustand cart/wishlist actions fire Kafka events (add_to_cart, add_to_wishlist, product_view, etc.) via server actions (actions/track-user.ts) to the user-events topic.
• Realtime: WebSocket provider for chatting service unread counts.

seller-ui (port 3000) — Seller Dashboard

• Stack: Next.js 16.1.6, React 19.2.3, Tailwind v4, zustand (authStore), react-hook-form, @tanstack/react-table, sonner.
• Pages: /dashboard (store performance KPIs + 6-month revenue chart), orders (status tabs + CSV export), create-product (large form with media/color/size/custom-spec editors + AI "Wand2" enhancement), all-products, edit-product, events CRUD, discount codes, customers, payments, invoices, inbox (WebSocket unread badges), notifications, app-platform, content, team, settings (custom domain, withdraw method).
• Key features: multi-store switcher, useSeller hook with switchShop mutation.

admin-ui (port 3002) — Admin Console

• Stack: Next.js 15.1.8, React 19, Tailwind v4, jotai (sidebar state), react-query + react-table, recharts, react-apexcharts, react-simple-maps (geographic).
• Pages: Dashboard (revenue/orders/sellers/customers KPIs, revenue chart, device pie chart, world map, recent orders), products, orders, order detail, payments, users (ban action), sellers, events, loggers (WebSocket at :6008), management, notifications, customization.

ghast-chat-web (port 3100) — Chat Client

• Stack: Next.js 14, React 18, Tailwind v3, zustand, native fetch + WebSocket.
• Single-page WhatsApp-style chat against the Go service (NEXT_PUBLIC_CHAT_API_URL || http://localhost:8090). Demo user via sessionStorage; no real auth.

website-builder (port 3004) — AI Storefront Builder

• Stack: Next.js 14.2.16, React 18, Tailwind v3, local shadcn/Radix UI suite, framer-motion, recharts, react-dnd, react-resizable-panels, zod + react-hook-form.
• Pages: / marketing landing, /login, /dashboard (website manager), /dashboard/templates (8 filterable templates), /dashboard/builder/[id] (drag-drop multi-page builder with device preview, undo/redo history, keyboard shortcuts, properties panel, HTML export), /dashboard/shopify-apps, /shopify/admin-embed, /shopify/storefront-demo.
• AI Builder: /api/ai/builder calls OpenAI gpt-4o-mini (strict JSON-schema output) returning a BuilderAgentPlan (add/rename/delete page/section/element ops) that a pure client-side engine (lib/builder-agent.ts) applies to the document. Needs OPENAI_API_KEY.
• Shopify simulation: full install → OAuth → session → admin embed → storefront extension → webhooks flow backed by an in-memory runtime (lib/shopify-sim/runtime.ts).

4. Backend Services (details)

auth-service (:6001)

Express + JWT + Swagger (/api-docs). Three identities: user, seller, admin.

• Auth flow: OTP-based registration + verification, JWT access (15 min) / refresh (7 days) tokens in httpOnly cookies, forgot/reset password, change password.
• Seller: complete profile, create shop, switch active shop, Stripe Connect account links.
• User: shipping address CRUD.
• Emails via nodemailer + EJS templates (activation, forgot password). OTP storage in Redis. Swagger docs auto-generated (swagger-output.json).

product-service (:6002)

Catalog + seller commerce tools. Uses relative imports of shared packages.

• Public: categories, all products (top-10 latest / top-sales), events, product by slug, filtered products/offers/shops (price/category/color/size/country), search, top shops.
• Seller (protected): discount-code CRUD (percentage/flat, shop-scoped), product CRUD with ImageKit upload (base64 → ImageKit /products folder), slug validation, soft delete with 24h restore window, variant attributes, events via start/end dates, Stripe account retrieval, per-product analytics.

seller-service (:6003)

• Shops: soft-delete (permanent after 28 days via node-cron job), restore, avatar/image upload, profile edit.
• Shop websites: CRUD (shopWebsites: slug, pages, theme, settings, SEO), publish (snapshot), archive; custom domains with TXT verification tokens, primary-domain switching.
• Public storefront resolution by domain, follow/unfollow, notifications (list + mark-as-read), shop members with roles (owner/manager/staff). Swagger at /api-docs.

order-service (:6004)

• Payments: Stripe PaymentIntents with 10% platform fee + seller Connect transfer; payment sessions in Redis (10-min TTL); raw-body webhook POST /api/create-order handling payment_intent.succeeded → creates per-shop orders, decrements stock, bumps totalSales + analytics, sends confirmation email, creates notifications.
• Order queries: per-user, per-seller (active shop), per-admin, detail (items + products + address + coupon).
• Delivery status: Ordered → Packed → Shipped → Out for Delivery → Delivered.
• Coupons: verification + percentage/flat discount computation.

admin-service (:6005)

Admin-only CRUD: products, events, admins (promote user), users, sellers (with first shop), site config, notifications. All routes require isAuthenticated + isAdmin (except public site-config and user notifications).

chatting-service (:6006)

Express REST + WebSocket on one HTTP server + Kafka consumer.

• WebSocket manages online presence (Redis online:user:* / online:seller:*), live message relay, MARK_AS_SEEN, unseen counts; publishes to Kafka chat.new_message.
• Batched Kafka consumer (flush every 3s) persists messages to MongoDB + increments Redis unseen counters.
• REST: create/find 1:1 conversation, user/seller conversation lists (with last message, online status, unread count), paginated messages per conversation (10/page).

recommendation-service (:6007)

• TensorFlow.js (CPU backend) collaborative-filtering-style model: 2-input embedding (user × product, 50-dim), dot-product + sigmoid, weighted labels (purchase 1.0 / add_to_cart 0.7 / add_to_wishlist 0.5 / product_view 0.1).
• Trains per request; cached for 3h per user once ≥50 actions exist; cold-start fallback to last 10 products. Returns top 10 recommendations.

kafka-service (no HTTP)

Background worker consuming user-events (group user-events-group), buffered 3s, validating actions (add_to_wishlist, add_to_cart, product_view, remove_from_cart, remove_from_wishlist). Updates userAnalytics (actions capped at 100, country/city/device) and productAnalytics (views/cartAdds/wishListAdds/purchases).

logger-service (:6008)

Kafka consumer (group log-events-group) on topic logs; buffers and broadcasts every 3s to all connected WebSocket logger clients (used by admin-ui /dashboard/loggers).

oAuth-service (:4000)

Minimal OAuth 2.0: GET /oauth/authorize issues 10-min codes, POST /oauth/token exchanges code for JWT access (1h) + refresh tokens, GET /oauth/userinfo returns user from bearer token. MongoDB collections: oauth_clients, oauth_codes, oauth_tokens.

ghast-chat-service (Go, :8090)

• Only non-JS service. Stdlib net/http, gorilla/websocket, lib/pq (Postgres), go-redis/v9.
• WebSocket rooms per conversation, REST: POST /api/v1/messages/send, GET /api/v1/messages?conversation_id=, POST /api/v1/conversations/create, GET /api/v1/conversations.
• Postgres (ghast_chat DB) with embedded migration runner; Redis queueing. Auth via X-User-ID header (demo user model). Graceful shutdown, multi-stage Dockerfile.

shopify-app-service (:3005, Next.js 16)

Standalone Shopify app simulator ("Atlas Flow Ops"). In-memory state (global singleton, POST /api/shopify/reset clears):

• /api/shopify/install, /oauth/authorize, /oauth/callback (HMAC + state verified), /session, /admin/context (embedded iframe), /example-app (mock Admin API), /webhooks/order-created (signature verified), /simulate/order-created (full round-trip), /storefront/extension, /events timeline.
• Pages: /dashboard, /embedded, /storefront.

5. Shared Packages

@repo/db — Prisma client

Singleton PrismaClient (global-cache pattern, MongoDB provider). Schema source of truth at packages/db/prisma/schema.prisma; no migration history — MongoDB schema-first db push workflow (prisma generate in CI/Docker). Root pnpm generate / pnpm studio.

Data models (25 + 2 enums): images, users, shopReviews, followers, shops, shopWebsites, websiteDomains, shopMembers, site_config, discount_codes, products (+productStatus enum), userAnalytics, productAnalytics, shopAnalytics, uniqueShopVisitors, orders, orderItems, address (+addressType enum), notifications, conversationGroup, participant, message, oauth_codes, oauth_clients, oauth_tokens.

Notable schema facts: all IDs are @db.ObjectId mapped to _id; products has unique slug, variant colors/sizes arrays, custom_specifications/custom_properties JSON, soft-delete flags; orders carry deliveryStatus; chat message has a composite unique key; websiteDomains has a unique domain + TXT verification fields; site_config holds global categories/subCategories/logo/banner (seeded by api-gateway on boot).

@repo/error-handler

Zero-dep. AppError base + NotFoundError (404), ValidationError (400), AuthError (401), ForbiddenError (403), DatabaseError (500), RateLimitError (429), plus errorMiddleware (returns { status, message, details? }).

@repo/middleware

• isAuthenticated: verifies JWT (Bearer header / access_token / seller-access-token cookies) via ACCESS_TOKEN_SECRET; loads user/seller + shops; resolves active shop from cookie/header/query/body; attaches req.user/req.seller.
• authorizeRoles: isSeller, isAdmin, isUser.

@workspace/ui

shadcn/ui (radix-nova style) design system on Tailwind CSS v4 + Radix UI + CVA + lucide-react.

• Primitives: Button, Card, Badge, Avatar, Accordion, Breadcrumb, Checkbox, Collapsible, Dialog, DropdownMenu, Input, Label, ScrollArea, Select, Separator, Sheet, Skeleton, Sonner Toaster, Tabs, Table, Textarea, Toggle, ToggleGroup, Tooltip, Drawer (vaul).
• Charts: recharts wrapper (ChartContainer, ChartAreaInteractive), SectionCards KPIs, DataTable (tanstack + dnd-kit).
• Sidebar system: Sidebar/AppSidebar, NavMain/Projects/Secondary/Documents/User, TeamSwitcher, SiteHeader, ThemeToggle.
• Globals CSS with OKLCH light/dark tokens; exports cn() util and useIsMobile() hook.

@atlas/imagekit

Preconfigured ImageKit client (hardcoded endpoint https://ik.imagekit.io/shahriarpenderghast).

@repo/redis

ioredis client (test-aware: in-memory stub when NODE_ENV=test). Helpers incrementUnseenCount / getUnseenCount / clearUnseenCount on keys unseen:<user|seller>_<conversationId>.

@atlas/kafka

kafkajs client (clientId: "eshop-kafka-client"); brokers resolve kafka:29092 in Docker/prod vs localhost:9092 locally.

@atlas/logs

sendLog({ type, message, source }) → publishes JSON to Kafka logs topic.

6. Event Streaming & Analytics

• Kafka topics (bootstrapped by one-shot kafka-setup job in compose / K8s): user-events, logs, chat.new_message.
• Pipeline: web storefront (zustand actions) → server action producer → user-events → kafka-service consumer → Mongo userAnalytics / productAnalytics → recommendation-service (TensorFlow.js) → personalized "Recommended for you".
• Chat: chatting-service WebSocket → chat.new_message → batched consumer → Mongo + Redis unseen counts.
• Logs: @atlas/logs sendLog → logs topic → logger-service → WebSocket broadcast to admin loggers console.
• userAnalytics also tracks country/city/device (via useDeviceTracking / useLocationTracking hooks in web) and unique shop visitors (uniqueShopVisitors).

7. AI Features

1. AI Builder (production) — in website-builder: OpenAI gpt-4o-mini route handler with strict JSON-schema output producing a BuilderAgentPlan (add/rename/delete page/section/element). Applied client-side by lib/builder-agent.ts. Chat panel UI ("Agentic" badge) with suggestion chips; configurable model via OPENAI_BUILDER_MODEL.
2. Recommendation engine (production) — TensorFlow.js embedding model per user (see recommendation-service).
3. AI/ sandbox (learning) — mini-GPT from scratch in PyTorch (trained weights checked in), char-level tokenizer, TF-IDF + cosine RAG, a "Copilot Lite" code-completion REPL, plus small nn/rl/bigram/trigram/gradient examples.

8. Infrastructure & DevOps

Docker Compose

• docker-compose.dev.yml: postgres:16, redis:7, zookeeper + kafka (confluent 7.4.0, kafka-setup creates topics), ghast-chat-service (:8090) and ghast-chat-web (:3100).
• docker-compose.production.yml: full platform — postgres, redis (password), zookeeper, kafka, kafka-setup, nginx (:80/:443, TLS via letsencrypt), and images for api-gateway, auth, product, order, seller, admin, chatting, logger, recommendation, kafka-service, user-ui (legacy name for web), seller-ui, admin-ui, ghast-chat-service, ghast-chat-web. All image tags ${DOCKER_USERNAME}/<service>:latest; network eshop-network.
• Root scripts: docker:prod, docker:build, docker:run, docker:stop, docker:clean, docker:logs — but scripts/local-production.sh is not committed (lives on the deploy host only).

Kubernetes (Kustomize) — devops/kubernetes/base

• Namespace atlas-commerce; atlas-platform-config ConfigMap (NODE_ENV, Kafka, DB, Redis, Next.js public URLs); atlas-app-secrets.example.yaml (real secrets not committed).
• Infra: PostgreSQL 16 (20Gi PVC), Redis 7, Zookeeper + Kafka (Confluent 7.4.0), kafka-topic-bootstrap Job.
• Workloads: deployments for api-gateway, auth/product/seller/order/admin/chatting/recommendation services, kafka-service, logger-service, web/seller-ui/admin-ui/ghast-chat-web, ghast-chat-service. Images are docker.io/replace-me/... placeholders.
• Ingress: api.atlas.local → gateway, app.atlas.local → web, seller.atlas.local → seller-ui, admin.atlas.local → admin-ui, chat.atlas.local → ghast-chat-web.

Terraform — devops/terraform/environments/dev

Baseline AWS dev platform (not a full prod landing zone):

• VPC module v5.19.0 (public/private subnets, 1 NAT gateway), EKS module v20.36.0 (Kubernetes 1.32, 1 managed node group t3.large, desired 3/min 2/max 6), AWS provider ~> 5.95, Terraform >= 1.8.0.
• Default region ap-south-1, VPC CIDR 10.42.0.0/16. No remote state / IAM separation / observability (documented TODO).

CI/CD

• GitHub Actions (.github/workflows/docker-build.yml): validate-devops (terraform fmt/validate + kustomize render) → detect-changes (per-service file mapping, force-rebuild/shared-change triggers) → build-backend/build-frontend matrices (Buildx, skip-if-exists, prisma generate) → deploy-to-ec2 (SSH to EC2_HOST, pulls repo, runs ./scripts/deploy-production.sh — script not in repo) → notify-completion.
• Jenkins (devops/jenkins/Jenkinsfile): equivalent pipeline with build parameters (FORCE_REBUILD, RUN_K6_SMOKE, APPLY_TERRAFORM, DEPLOY_TO_K8S, DOCKERHUB_NAMESPACE) and a detect-changed-services.sh helper.

Load Testing (k6) — devops/k6

• smoke.js — 1 VU uptime check on /gateway-health (p95 < 800ms, failure < 1%).
• gateway-ramp.js — ramp 20→80→120 VUs over ~10 min (p95 < 1200ms, failure < 2%).
• checkout-spike.js — spike to 150 VUs against /api/orders (p95 < 1800ms, failure < 3%).
• ui-browse.js — browser navigation across web (:3001), seller-ui (:3000), admin-ui (:3002).
• Run via root scripts devops:k6:*.

9. Environment Variables (names only)

DATABASE_URL (MongoDB) · REDIS_URL · REDIS_DATABASE_URI / UPSTASH_REDIS_REST_URL / UPSTASH_REDIS_REST_TOKEN (Upstash) · OPENAI_API_KEY · SMTP_HOST/SERVICE/PORT/USER/PASS (Gmail) · NODE_ENV · ACCESS_TOKEN_SECRET / REFRESH_TOKEN_SECRET (JWT) · IMAGEKIT_PUBLIC_KEY / IMAGEKIT_SECRET_KEY · NEXT_PUBLIC_SERVER_URI (gateway URL) · KAFKA_BROKERS · NEXT_PUBLIC_CHATTING_WEBSOCKET_URI · NEXT_PUBLIC_STRIPE_PUBLIC_KEY / STRIPE_SECRET_KEY · NEXT_PUBLIC_FRONTEND_URL · DOCKER_ENV, PORT, DOCKER_USERNAME (compose/deploy).

10. Common Developer Workflows

pnpm install # install all workspace deps
pnpm generate # prisma generate (MongoDB client)
pnpm dev # turbo dev --concurrency=18 (all apps + services)
pnpm build / pnpm lint # turbo build / lint
pnpm studio # Prisma Studio

# Infra (local dev)
docker compose -f docker-compose.dev.yml up -d # kafka, zookeeper, postgres, redis, ghast chat
docker compose -f docker-compose.production.yml up kafka zookeeper -d # kafka only

# Ghast chat (standalone)
bash scripts/start-ghast-chat.sh # or start-ghast-chat.bat on Windows

# Load tests
pnpm devops:k6:smoke / ramp / checkout / ui

# K8s / Terraform
kubectl kustomize devops/kubernetes/base
terraform -chdir=devops/terraform/environments/dev init -backend=false && terraform -chdir=devops/terraform/environments/dev validate

11. Known Gaps & Caveats

• Missing scripts in repo: scripts/local-production.sh (used by 6 root docker scripts) and scripts/deploy-production.sh (used by CI deploy) — they live on the EC2 host only.
• Naming drift: production compose uses user-ui while the repo app is web; nginx.conf referenced by production compose is absent from the repo.
• Placeholders: Kubernetes images docker.io/replace-me/..., Jenkins DOCKERHUB_NAMESPACE=replace-me.
• Shipping service is an empty placeholder directory.
• Credentials hygiene: @repo/redis hardcodes an Upstash URL (with creds) as fallback; @atlas/imagekit hardcodes its endpoint; docker-compose.production.yml commits a Stripe public key and prod URLs. .env (gitignored) contains live credentials.
• Terraform is a dev-only baseline (no remote state, IAM separation, or observability stack).
• Root README is still the stock shadcn monorepo template (this summary.md is the real project documentation).