dagster
https://github.com/dagster-io/dagster
Python
An orchestration platform for the development, production, and observation of data assets.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported8 Subscribers
Add a CodeTriage badge to dagster
Help out
- Issues
- GCS equivalent to get_s3_keys
- Example of submitting a pyspark job to dataproc
- expose output managers
- PartitionedIOManager abstract base class
- context manager for grouping asset definitions
- "notebook" view of a job or set of assets
- Add a metadata value for noneable types
- Be able to set run tags at repo level
- EmrPySparkStepLauncher enters an invalid state if an error occurs fetching logs after the launched step succeeds
- support additional container overrides in the EcsRunLauncher beyond just CPU and memory (like ephemeralStorage)
- Docs
- Python not yet supported