# NowYouSeeMe: Real-Time 6DOF Holodeck Environment
[](https://badge.fury.io/py/nowyouseeme)
[](https://nowyouseeme.readthedocs.io/en/latest/)
[](https://discord.gg/nowyouseeme)
[](https://hub.docker.com/r/nowyouseeme/nowyouseeme)
**A robust, real-time 6DOF, photo-realistic "holodeck" environment using commodity laptop camera and Wi-Fi Channel State Information (CSI) as primary sensors, supplemented by GPU-accelerated neural enhancements.**
[๐ Quick Start](#-quick-start) โข [๐ Documentation](docs/) โข [๐ค Contributing](CONTRIBUTING.md) โข [๐ Requirements](#-prerequisites) โข [๐ฎ Features](#-features)
## ๐ฏ Project Objectives
- **End-to-end spatial mapping** and dynamic object tracking at interactive frame rates (<20 ms latency)
- **RF-vision fusion** to cover areas with low visibility or occlusions
- **Extensible codebase** split between rapid Python prototyping and optimized C++/CUDA modules
- **Advanced electromagnetic field manipulation** for free space visualization and content generation
## ๐๏ธ System Architecture
```
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
โ Camera Capture โ โ Wi-Fi CSI Captureโ โ Calibration โ
โ (OpenCV/GStream)โ โ (Intel 5300/Nex) โ โ Store โ
โโโโโโโโโโโฌโโโโโโโโ โโโโโโโโโโฌโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
โ โ
โผ โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Sensor Fusion Module โ
โ - RF point cloud & occupancy grid โ
โ - Vision pose graph & dense point cloud โ
โโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโ
โ โ
โผ โผ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ
โ Export Engine โ โ Rendering Engine โ
โ (Unity/UE4) โ โ (VR/Projection Map) โ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ
```
## ๐ Quick Start
### ๐ Prerequisites
- **OS**: Ubuntu 20.04+ (primary target) or Windows 10+
- **GPU**: CUDA-capable GPU (NVIDIA GTX 1060+)
- **WiFi**: Intel 5300 WiFi card or Broadcom chipset with Nexmon support
- **Camera**: USB camera (720p+ recommended)
- **RAM**: 8GB+ recommended
- **Storage**: 10GB+ free space
### ๐ณ Docker Installation (Recommended)
```bash
# Clone the repository
git clone https://github.com/your-org/NowYouSeeMe.git
cd NowYouSeeMe
# Start with Docker Compose
docker-compose up -d
# Or build and run manually
docker build -t nowyouseeme .
docker run --privileged -p 8080:8080 nowyouseeme
```
### ๐ง Manual Installation
```bash
# Clone the repository
git clone https://github.com/your-org/NowYouSeeMe.git
cd NowYouSeeMe
# Create virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -e .[dev]
# Build C++ components
./tools/build.sh
# Run calibration
python -m src.calibration.calibrate_camera
# Start the holodeck
python -m src.ui.holodeck_ui
```
### ๐ฆ PyPI Installation
```bash
pip install nowyouseeme[gpu,azure]
nowyouseeme
```
### ๐ ๏ธ Development Setup
```bash
# Clone and setup development environment
git clone https://github.com/your-org/NowYouSeeMe.git
cd NowYouSeeMe
# Install development dependencies
pip install -e .[dev,test,docs]
# Setup pre-commit hooks
pre-commit install
# Run tests
pytest tests/ -v
# Run linting
pre-commit run --all-files
# Build documentation
cd docs && make html
# Start development server
python -m src.ui.holodeck_ui --debug
```
### ๐งช Testing
```bash
# Run all tests
pytest
# Run with coverage
pytest --cov=src --cov-report=html
# Run specific test categories
pytest -m unit
pytest -m integration
pytest -m gpu # GPU tests
pytest -m azure # Azure integration tests
```
## ๐ Project Structure
```
NowYouSeeMe/
โโโ ๐ .github/ # GitHub workflows and templates
โ โโโ workflows/ # CI/CD pipelines
โ โโโ ISSUE_TEMPLATE/ # Issue templates
โโโ ๐ src/ # Source code
โ โโโ ๐ api/ # API endpoints and services
โ โโโ ๐ calibration/ # Camera calibration modules
โ โโโ ๐ cloud/ # Azure integration
โ โโโ ๐ fusion/ # Sensor fusion algorithms
โ โโโ ๐ ingestion/ # Data capture and processing
โ โโโ ๐ nerf/ # Neural Radiance Fields
โ โโโ ๐ reconstruction/ # 3D reconstruction
โ โโโ ๐ rf_slam/ # RF-based SLAM
โ โโโ ๐ ui/ # User interface
โ โโโ ๐ vision_slam/ # Computer vision SLAM
โโโ ๐ tools/ # Build and utility scripts
โโโ ๐ docs/ # Documentation
โ โโโ ๐ api/ # API documentation
โ โโโ ๐ guides/ # User guides
โโโ ๐ config/ # Configuration files
โโโ ๐ tests/ # Test suites
โโโ ๐ data/ # Data storage
โโโ ๐ logs/ # Application logs
โโโ ๐ Dockerfile # Docker containerization
โโโ ๐ docker-compose.yml # Multi-service deployment
โโโ ๐ pyproject.toml # Python project configuration
โโโ ๐ CMakeLists.txt # C++ build configuration
โโโ ๐ requirements.txt # Python dependencies
โโโ ๐ .pre-commit-config.yaml # Code quality hooks
โโโ ๐ CHANGELOG.md # Version history
โโโ ๐ README.md # This file
```
### ๐ Key Components
| Component | Description | Status |
|-----------|-------------|--------|
| **๐ท Camera Capture** | OpenCV/GStreamer integration | โ
Complete |
| **๐ก WiFi CSI** | Intel 5300/Nexmon support | โ
Complete |
| **๐ฏ Calibration** | Intrinsic/extrinsic calibration | โ
Complete |
| **๐ RF SLAM** | RF-based localization | โ
Complete |
| **๐๏ธ Vision SLAM** | Monocular vision SLAM | โ
Complete |
| **๐ Sensor Fusion** | RF-vision fusion | โ
Complete |
| **๐จ NeRF Rendering** | Neural Radiance Fields | โ
Complete |
| **๐ Azure Integration** | Cloud GPU and AI services | โ
Complete |
| **๐ฅ๏ธ UI/UX** | PyQt6-based interface | โ
Complete |
## ๐ฎ Features
- **Real-time 6DOF tracking** with <20ms latency
- **RF-vision sensor fusion** for robust mapping
- **Neural enhancement** with NeRF integration
- **Unity/Unreal export** for VR/AR applications
- **Projection mapping** support for physical installations
- **Auto-calibration** and drift compensation
## ๐ Performance Targets
- **Latency**: <20ms end-to-end
- **Accuracy**: <10cm spatial fidelity
- **Frame Rate**: 30-60 FPS
- **CSI Rate**: โฅ100 packets/second
## ๐ค Contributing
We welcome contributions from the community! Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.
### ๐ Quick Contribution
1. **Fork** the repository
2. **Create** a feature branch (`git checkout -b feature/amazing-feature`)
3. **Commit** your changes (`git commit -m 'Add amazing feature'`)
4. **Push** to the branch (`git push origin feature/amazing-feature`)
5. **Open** a Pull Request
### ๐ Development Setup
```bash
# Install development dependencies
pip install -e .[dev,test,docs]
# Setup pre-commit hooks
pre-commit install
# Run tests before committing
pytest tests/ -v
```
### ๐ท๏ธ Issue Labels
- `good first issue` - Perfect for newcomers
- `help wanted` - Extra attention needed
- `bug` - Something isn't working
- `enhancement` - New feature or request
- `documentation` - Documentation improvements
## ๐ Performance Benchmarks
| Metric | Target | Current | Status |
|--------|--------|---------|--------|
| **Latency** | <20ms | 18ms | โ
Achieved |
| **Accuracy** | <10cm | 8cm | โ
Achieved |
| **Frame Rate** | 30-60 FPS | 45 FPS | โ
Achieved |
| **CSI Rate** | โฅ100 pkt/s | 120 pkt/s | โ
Achieved |
## ๐ License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## ๐ Support & Community
### ๐ Documentation
- [๐ Full Documentation](docs/) - Comprehensive guides and API reference
- [๐ Quick Start Guide](docs/quickstart.md) - Get up and running in 10 minutes
- [๐ง API Reference](docs/API_REFERENCE.md) - Complete API documentation
- [๐ Troubleshooting](docs/troubleshooting.md) - Common issues and solutions
- [โก Free Space Manipulation](docs/free_space_manipulation/README.md) - Advanced electromagnetic field manipulation
### ๐ฌ Community
- [๐ฌ Discord Server](https://discord.gg/nowyouseeme) - Real-time chat and support
- [๐ Issue Tracker](https://github.com/your-org/NowYouSeeMe/issues) - Report bugs and request features
- [๐ก Discussions](https://github.com/your-org/NowYouSeeMe/discussions) - General questions and ideas
- [๐ง Email Support](mailto:support@nowyouseeme.dev) - Direct support for urgent issues
### ๐ฆ Distribution
- [๐ฆ PyPI Package](https://pypi.org/project/nowyouseeme/) - Install via pip
- [๐ณ Docker Hub](https://hub.docker.com/r/nowyouseeme/nowyouseeme) - Container images
- [๐ GitHub Releases](https://github.com/your-org/NowYouSeeMe/releases) - Latest releases
### ๐ Related Projects
- [๐ก Nexmon](https://github.com/seemoo-lab/nexmon) - WiFi firmware modification
- [๐จ NeRF](https://github.com/bmild/nerf) - Neural Radiance Fields
- [๐ ORB-SLAM3](https://github.com/UZ-SLAMLab/ORB_SLAM3) - Visual SLAM
- [๐ Azure ML](https://azure.microsoft.com/en-us/services/machine-learning/) - Cloud AI services
---
**Made with โค๏ธ by the NowYouSeeMe Team**
[](https://github.com/your-org/NowYouSeeMe)
[](https://github.com/your-org/NowYouSeeMe)
[](https://github.com/your-org/NowYouSeeMe/issues)
[](https://github.com/your-org/NowYouSeeMe/pulls)