dagster
https://github.com/dagster-io/dagster
Python
An orchestration platform for the development, production, and observation of data assets.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported8 Subscribers
Add a CodeTriage badge to dagster
Help out
- Issues
- Spark Connect support for dagster
- Possibility to union partitions of multiple PartitionDefinitions
- Ability to declaratively set global concurrency key limits
- Add group name for InputContext and OutputContext
- Expose path of dagster dev temporary directory
- Allow adding output metadata from `InitResourceContext`
- Emit dynamic partition entries when testing sensors in a branch deployment
- Ability to block certain partitions/backfills from executing
- Retrieve the failed assets in a run.
- gRPC servers spawned by the dagster-daemon seem to create zombie processes
- Docs
- Python not yet supported