dagster
https://github.com/dagster-io/dagster
Python
An orchestration platform for the development, production, and observation of data assets.
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
Python not yet supported6 Subscribers
Add a CodeTriage badge to dagster
Help out
- Issues
- The dagster_k8s.execute_k8s_job function fails when trying to retrieve pod logs of pod containing multiple containers.
- Support Kafka Consumer message queue to handle streaming pipeline
- Global Asset Lineage graph positions assets in suboptimal way causing connectors to cross each other
- Respect partition mappings in asset job backfills
- monte carlo data observability tool integration
- Vertex AI Job Executor
- How use Datahub w/ Dagster?
- Jobs/Backfills support same partition definition with different start dates
- Ability to save files with specific metadata using the S3 IO Manager
- Allow to pass capacityProviderStrategy in run_task_kwargs in order to enable FARGATE_SPOT (AWS ECS)
- Docs
- Python not yet supported