All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[Unreleased]
Added
- Daemon Mode (HTTP Server): Run SlimJSON as an HTTP service
-dor-daemonflag to run as HTTP server-portflag to specify port (default: 8080)- RESTful API with 3 endpoints:
POST /slim?profile=<name>- Compress JSONGET /health- Health checkGET /profiles- List available profiles
- Supports all built-in and custom profiles
- Production-ready HTTP server
- Custom Config File Priority:
-cor-configflag- Specify custom config file path
- Takes priority over
.slimjsonsearch - Enables multiple config files for different environments
- Default Help Message: Shows usage when run without arguments
- Comprehensive help with all options
- Examples for common use cases
- API documentation for daemon mode
- API Documentation: Complete REST API documentation
- OpenAPI 3.0 specification (Swagger)
- API reference with examples
- Integration guides for Python, JavaScript, Go
- Docker and Kubernetes deployment examples
- Package Documentation: Comprehensive Go package documentation
- Full API reference for pkg.go.dev
- Usage examples and best practices
- Real-world integration patterns
- Thread safety guarantees
- HTTP API Tests: Complete test suite for daemon mode
- Health endpoint tests
- Profiles endpoint tests
- Compression endpoint tests with multiple scenarios
- Error handling tests
- Config priority tests
- Automated Release Pipeline: GitHub Actions for publishing
- Automatic binary builds for multiple platforms (linux, darwin, freebsd × amd64, arm64)
- GitHub Releases with changelog and checksums
- Automatic publication to pkg.go.dev
- Docker image builds and push to ghcr.io
- Multi-architecture Docker support (amd64, arm64)
- Documentation validation workflow
- Release process documentation (RELEASING.md)
- Configuration File Support:
.slimjsonconfiguration file for custom profiles- Searches in current directory and user home directory
- Simple INI-style format:
[profile-name]followed bykey=valuepairs - Custom profiles take precedence over built-in profiles
- Supports all compression parameters
- See
.slimjson.examplefor complete reference
- CLI Predefined Profiles: New
-profileflag with 4 predefined compression profiles:light: Light compression, preserves most data (MaxDepth: 10, MaxListLength: 20)medium: Balanced compression (MaxDepth: 5, MaxListLength: 10)aggressive: Removes verbose text fields (MaxDepth: 3, MaxListLength: 5, BlockList: description, summary, etc.)ai-optimized: Removes URLs and metadata (MaxDepth: 4, MaxListLength: 8, BlockList: *_url fields)
Basic Optimizations:
- Numeric Precision Control:
-decimal-places Nrounds floats to N decimal places (e.g., 19.999 → 20.0) - Array Deduplication:
-deduplicateremoves duplicate values from arrays - Array Sampling Strategies:
-sample-strategywith 4 modes:none: No sampling (default)first_last: Keep first N/2 and last N/2 elementsrandom: Random N elementsrepresentative: Evenly distributed sampling
- Sample Size Control:
-sample-size Nspecifies number of items to keep when sampling - String Truncation with Ellipsis: Truncated strings now end with
...to indicate content was cut
Advanced Compression Features:
- Null Compression:
-null-compressiontracks removed null fields in_nullsarray - Type Inference:
-type-inferenceconverts uniform arrays to schema+data format (40% savings) - Boolean Compression:
-bool-compressionconverts booleans to bit flags (60% savings) - Timestamp Compression:
-timestamp-compressionconverts ISO timestamps to unix timestamps (50% savings) - String Pooling:
-string-poolingdeduplicates repeated strings (30-50% savings)-string-pool-min Nsets minimum occurrences (default: 2)
- Number Delta Encoding:
-number-deltauses delta encoding for sequential numbers (50% savings)-number-delta-threshold Nsets minimum array size (default: 5)
- Enum Detection:
-enum-detectionconverts repeated categorical values to enums (20-40% savings)-enum-max-values Nsets maximum unique values (default: 10)
Changed
- Profiles no longer truncate strings to preserve data integrity - use BlockList instead to remove entire unnecessary fields
- Profile flags can be overridden: Use
-profile medium -decimal-places 2to combine profile with custom settings - Comprehensive compression testing suite in
testing/directory - Three real-world JSON test files (resume.json, schema-resume.json, users.json)
- Compression benchmark tool (
compression_benchmark.go) with detailed metrics - Token counting in compression tests - shows estimated token reduction for AI/LLM use cases
- Statistical analysis with standard deviation calculation (10 iterations per test)
- Detailed methodology documentation in
testing/METHODOLOGY.md - Performance benchmarks (
slimjson_bench_test.go) with 9 different scenarios - Compression results table in README showing real-world reduction percentages
- Four compression profiles: Light, Medium, Aggressive, AI-Optimized
- Performance metrics documentation (16-47µs processing time)
- Testing documentation in
testing/README.md - Makefile targets:
make benchandmake compression-test - Shell script
run_tests.shfor easy test execution - Documentation files: EXAMPLES.md, QUICK_START.md, TESTING_SUMMARY.md
- GitHub Pages with Jekyll and Cayman theme for documentation
- Automatic deployment via GitHub Actions
Changed
- Updated README with comprehensive compression results and benchmarks
- Enhanced .gitignore to exclude testing artifacts
- Improved Makefile with new testing and benchmarking targets
Performance
- Small files (5KB): ~16µs per operation
- Medium files (25KB): ~39µs per operation
- Large files (28KB): ~47µs per operation
- Excellent parallel processing scalability
[0.1.6] - Previous Release
Features
- Core JSON slimming functionality
- CLI tool with multiple configuration options
- Docker/Podman support
- Multi-platform binaries (Linux, macOS, FreeBSD)
- Basic unit tests