dagster
https://github.com/dagster-io/dagster
Python
An orchestration platform for the development, production, and observation of data assets.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported11 Subscribers
Add a CodeTriage badge to dagster
Help out
- Issues
- Dagster gRPC servers user a large memory footprint when loading jobs that have large dependencies
- allow supplying IO managers when building assets from graphs or ops
- "Upstream data" and "Code changed" indicators for partitioned assets
- Very slow job partitions page in Dagit UI for jobs with a high number of partitions
- Request to improve performance of writing thousands of DynamicOutputs via I/O manager (esp. to s3 from k8s / EKS)
- [Interest Poll] VSCode Extension / Local Dagster IDE
- Add function for accessing current environment
- SourceAssets don't support InputManagers
- PartitionMapping for last partition of asset depends unpartitioned asset and earlier partitions are uncomputable
- `graph_multi_asset` doesn't properly pass the descriptions of `AssetOut`s
- Docs
- Python not yet supported