dailz/hpc
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
103 Commits 1 Branch 0 Tags
4fd331ebd89ef208e39f1e9d758af9283d0c5896
Commit Graph
103 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
dailz 4fd331ebd8 feat(application): add Environment field and inject into Slurm job submission 2026-04-21 10:23:31 +08:00
dailzandSisyphus d9ca9233b3 fix(service): correct CPU/memory mapping and add TRES/memory_used extraction 
- Map CPUs to CpusPerTask (not MinimumCpus) for consistent SlurmDBD history

- Add Set:true to memory Uint64NoVal on submission

- Filter number=0 in mapUint64NoValToInt64 to avoid false zeros

- Extract peak memory from Steps.Tres.Requested.Max across all steps

- Add formatTresList, parseGresDetail, extractMemoryFromSteps helpers

- Update mapJobInfo and mapSlurmdbJob with new field mappings

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 17:10:19 +08:00
dailzandSisyphus d79656c728 feat(model): add resource fields to JobResponse (CPU, memory, TRES, elapsed) 
Add CpusPerTask, MemoryPerCpu, MemoryPerNode, MemoryUsed, TRES strings,

GresDetail, and Elapsed fields to capture full Slurm job resource info.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 17:09:52 +08:00
dailzandSisyphus 0a0e5fd7d4 fix(web): downgrade deps for TS 5.3 compatibility 
vue-tsc 1.8.27 -> 2.0.24 (monkey-patching broke with TS 5.3).
Remove erasableSyntaxOnly, ignoreDeprecations (TS 6.0 only).
Remove verbatimModuleSyntax, target es2023 -> es2022.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 13:41:46 +08:00
dailzandSisyphus 22963d0e6f fix(web): map selected files to file-type params and hide auto-injected params 
File/directory type app parameters now show the associated file name instead of
a disabled input. WORK_DIR is hidden from the form as it is auto-injected by the
backend. Selected files are mapped to file params via fileParamMapping on submit.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 13:41:37 +08:00
dailzandSisyphus 08ca4da691 fix(service): inject WORK_DIR and map file_ids before param validation 
Previously ValidateParams ran before WORK_DIR injection and file_ids mapping,
causing required parameter missing errors for auto-handled params. Now the
execution order is: inject WORK_DIR, map file_ids to file params, validate, resolve.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 13:41:28 +08:00
dailzandSisyphus a7c48dae84 test(web): add Playwright config and E2E tests for file manager 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:40:05 +08:00
dailzandSisyphus 6c19fed2f3 feat(web): add task views, file manager routing and sidebar menu 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:39:58 +08:00
dailzandSisyphus c197b9622b feat(web): add file manager views with chunked upload and file picker 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:39:48 +08:00
dailzandSisyphus f5e021d652 feat(web): add file manager and task types with API clients 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:39:38 +08:00
dailzandSisyphus d71cda3420 docs(openapi): update spec with task defaults and file_ids 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:39:23 +08:00
dailzandSisyphus 1ebf43355b test(server): add integration tests for task defaults and file_ids 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:39:16 +08:00
dailzandSisyphus 13ce86b1ef feat(mockserver): update mock server for file upload and task defaults 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:39:07 +08:00
dailzandSisyphus 9aea1ea710 feat(handler): add task defaults and file_ids support in task submission 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:38:59 +08:00
dailzandSisyphus e90904cedb test(service): add tests for task defaults and job status 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:38:49 +08:00
dailzandSisyphus 166ca3092c feat(service): add task defaults, job status, and cluster helpers 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:38:41 +08:00
dailzandSisyphus f894e870ed test(model): add tests for task defaults, job queries, and DTOs 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:38:30 +08:00
dailzandSisyphus db06e99967 feat(model): add task defaults, job queries, and refine file/task DTOs 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-20 10:37:58 +08:00
dailzandSisyphus 0c7a282386 feat(web): expose dev server to LAN for mock server testing 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-17 13:23:55 +08:00
dailzandSisyphus 9a04874847 feat(cmd/mockserver): add standalone mock server for frontend development 
Reuse MockSlurm + MockMinIO + TestEnv wiring pattern to create a standalone
binary that serves all API endpoints with in-memory SQLite and seed data.

- internal/mockserver/server.go: assembly logic (New/Close/Run), option pattern,
  4 accessors for seed data injection
- cmd/mockserver/main.go: CLI flags (--port, --seed, MOCK_PORT), 6 seed jobs
  in all 5 states + 2 seed applications, signal handling

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-17 13:21:13 +08:00
dailzandSisyphus a5ce173b54 feat(web): add Jobs pages - list, history, submit, detail 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 22:45:59 +08:00
dailzandSisyphus 1356f91b9d feat(web): add API client, router, and app layout 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 22:45:23 +08:00
dailzandSisyphus 01a1119d13 feat(web): scaffold Vue 3 + Vite + TypeScript project with Element Plus 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 22:44:40 +08:00
dailzandSisyphus 6a7bde4801 test(service): add tests for WORK_DIR injection and file-type param resolution 
- ValidateParams: file/directory type validation tests

- RenderScript: file-type not escaped, WORK_DIR injected without quotes

- ProcessTask: file_id→filename resolution, invalid ID, missing file scenarios

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 17:56:42 +08:00
dailzandSisyphus 07ae8ad6cd feat(service): auto-inject WORK_DIR and resolve file-type params in task script rendering 
- ProcessTask injects $WORK_DIR only when script template uses it

- File/directory type params: resolves file_id to filename before rendering

- ValidateParams validates file/directory params as valid int64 file IDs

- RenderScript no longer shell-escapes file/directory type values

- Log rendered script before submitting to Slurm for debugging

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 17:56:28 +08:00
dailzandSisyphus c1d0665b42 docs(openapi): add folder_path, user_id to FileResponse and user_id param to ListFiles 
- FileResponse schema: add folder_path (string, nullable), user_id (int64, nullable)

- FileResponse schema: folder_id now nullable instead of omitted

- GET /files: add user_id query parameter for filtering by owner

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 16:05:03 +08:00
dailzandSisyphus 29525c3fa9 feat(handler): update GetFile and ListFiles handlers for new FileResponse fields 
- GetFile uses new GetFileResponse instead of manual FileResponse construction

- ListFiles handler parses optional user_id query parameter

- Wire FolderStore into FileService in app.go, testenv, and file_test

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 16:04:43 +08:00
dailzandSisyphus 1d591efeba feat(model,service): add folder_path and user_id to FileResponse, add user_id filter to ListFiles 
- FileResponse gains folder_path ("/" for root) and user_id fields

- folder_id no longer uses omitempty, root files return null

- ListFiles accepts optional userID parameter for filtering by owner

- New buildFileResponse helper populates folder_path from FolderStore

- New GetFileResponse method wraps GetFileMetadata + buildFileResponse

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 16:04:22 +08:00
dailzandSisyphus 9092278d26 refactor(test): update test expectations for removed submit route 
- Comment out submit route assertions in main_test.go and server_test.go

- Comment out TestTask_OldAPICompatibility in task_test.go

- Update expected route count 31→30 in testenv env_test.go

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 15:16:07 +08:00
dailzandSisyphus 7c374f4fd5 refactor(handler,server): disable SubmitApplication endpoint, replaced by POST /tasks 
- Comment out SubmitApplication handler method

- Comment out route registration in server.go (interface + router + placeholder)

- Comment out related handler tests

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 15:15:55 +08:00
dailzandSisyphus 36d842350c refactor(service): disable SubmitFromApplication fallback, fully replaced by POST /tasks 
- Comment out SubmitFromApplication method and its fallback path

- Comment out 5 tests that tested the old direct-submission code

- Remove unused imports after commenting out the method

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 15:15:42 +08:00
dailzandSisyphus 80f2bd32d9 docs(openapi): update spec to match code — add Tasks, fix schemas, remove submit endpoint 
- Add Tasks tag, /tasks paths, and Task schemas (CreateTaskRequest, TaskResponse, TaskListResponse)

- Fix SubmitJobRequest.work_dir, InitUploadRequest mime_type/chunk_size, UploadSessionResponse.created_at

- Fix FolderResponse: add file_count/subfolder_count, remove updated_at

- Fix response wrapping for File/Upload/Folder endpoints to use ApiResponseSuccess

- Remove /applications/{id}/submit path and ApplicationSubmitRequest schema

- Update Applications tag description

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-16 15:15:31 +08:00
dailz 52a34e2cb0 feat(client): add CLI client entry point 2026-04-16 13:24:21 +08:00
dailz b9b2f0d9b4 feat(testutil): add MockSlurm, MockMinIO, TestEnv and 37 integration tests 
- mockminio: in-memory ObjectStorage with all 11 methods, thread-safe, SHA256 ETag, Range support
- mockslurm: httptest server with 11 Slurm REST API endpoints, job eviction from active to history queue
- testenv: one-line test environment factory (SQLite + MockSlurm + MockMinIO + all stores/services/handlers + httptest server)
- integration tests: 37 tests covering Jobs(5), Cluster(5), App(6), Upload(5), File(4), Folder(4), Task(4), E2E(1)
- no external dependencies, no existing files modified
 2026-04-16 13:23:27 +08:00
dailz 73504f9fdb feat(app): add TaskPoller, wire DI, and add task integration tests 2026-04-15 21:31:17 +08:00
dailz 3f8a680c99 feat(handler): add TaskHandler endpoints and register task routes 2026-04-15 21:31:11 +08:00
dailz ec64300ff2 feat(service): add TaskService, FileStagingService, and refactor ApplicationService for task submission 2026-04-15 21:31:02 +08:00
dailz acf8c1d62b feat(store): add TaskStore CRUD and batch query methods for files and blobs 2026-04-15 21:30:51 +08:00
dailz d46a784efb feat(model): add Task model, DTOs, and status constants for task submission system 2026-04-15 21:30:44 +08:00
dailzandSisyphus 79870333cb fix(service): tolerate concurrent pending-to-uploading status race in UploadChunk 
When multiple chunk uploads race on the pending→uploading transition, ignore ErrRecordNotFound from UpdateSessionStatus since another request already completed the update.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 10:27:12 +08:00
dailzandSisyphus d9a60c3511 fix(model): rename Application table to hpc_applications 
Avoid table name collision with other systems.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:32:11 +08:00
dailzandSisyphus 20576bc325 docs(openapi): add file storage API specifications 
Add 13 endpoints for chunked upload, file management, and folder CRUD with 6 new schemas.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:30:18 +08:00
dailzandSisyphus c0176d7764 feat(app): wire file storage DI, cleanup worker, and integration tests 
Add DI wiring with graceful MinIO fallback, background cleanup worker for expired sessions and leaked multipart uploads, and end-to-end integration tests.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:23:25 +08:00
dailzandSisyphus 2298e92516 feat(handler): add upload, file, and folder handlers with routes 
Add UploadHandler (5 endpoints), FileHandler (4 endpoints), FolderHandler (4 endpoints) with Gin route registration in server.go.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:23:17 +08:00
dailzandSisyphus f0847d3978 feat(service): add upload, download, file, and folder services 
Add UploadService (dedup, chunk lifecycle, ComposeObject), DownloadService (Range support), FileService (ref counting), FolderService (path validation).

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:23:09 +08:00
dailzandSisyphus a114821615 feat(server): add streaming response helpers for file download 
Add ParseRange, StreamFile, StreamRange for full and partial content delivery.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:22:58 +08:00
dailzandSisyphus bf89de12f0 feat(store): add blob, file, folder, and upload stores 
Add BlobStore (ref counting), FileStore (soft delete + pagination), FolderStore (materialized path), UploadStore (idempotent upsert), and update AutoMigrate.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:22:44 +08:00
dailzandSisyphus c861ff3adf feat(storage): add ObjectStorage interface and MinIO client 
Add ObjectStorage interface (11 methods) with MinioClient implementation using minio-go Core.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:22:33 +08:00
dailzandSisyphus 0e4f523746 feat(model): add file storage GORM models and DTOs 
Add FileBlob, File, Folder, UploadSession, UploadChunk models with validators.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:22:25 +08:00
dailzandSisyphus 44895214d4 feat(config): add MinIO object storage configuration 
Add MinioConfig struct with connection, bucket, chunk size, and session TTL settings.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-04-15 09:22:18 +08:00