Search Results for "parallel"
Sort By:
Showing 88 open source projects for "parallel"
View related business solutions-
1
golangci-lint
Fast linters Runner for Go
golangci-lint is a fast Go linters runner. It runs linters in parallel, uses caching, supports yaml config, has integrations with all major IDE and has dozens of linters included. ⚡ Very fast: runs linters in parallel, reuses Go build cache and caches analysis results. Yaml-based configuration. Integrations with VS Code, Sublime Text, GoLand, GNU Emacs, Vim, Atom, GitHub Actions. A lot of linters included, no need to install them. -
2
compress
Optimized Go Compression Packages
...Optimized deflate packages which can be used as a dropin replacement for gzip, zip and zlib. snappy is a drop-in replacement for github.com/golang/snappy offering better compression and concurrent streams. huff0 and FSE implementations for raw entropy encoding. gzhttp Provides client and server wrappers for handling gzipped requests efficiently. pgzip is a separate package that provides a very fast parallel gzip implementation. Stream decompression is now faster on asynchronous since the goroutine allocation much more effectively splits the workload. On typical streams, this will typically use 2 cores fully for decompression. When a stream has finished decoding no goroutines will be left over, so decoders can now safely be pooled and still be garbage collected. -
3
Apache Beam
Unified programming model for Batch and Streaming
Apache Beam is an open source, unified programming model to define both batch and streaming data-parallel processing pipelines, as well as certain language-specific SDKs for constructing pipelines and Runners. These pipelines are executed on one of Beam’s supported distributed processing back-ends, which include Apache Apex, Apache Flink, Apache Spark, and Google Cloud Dataflow. Beam is especially useful for Embarrassingly Parallel data processing tasks, and caters to the different needs and backgrounds of end users, SDK writers and runner writers. -
4
Piscina
A fast, efficient Node.js Worker Thread Pool implementation
Piscina is a fast, efficient Node.js worker thread pool implementation, designed to perform heavy CPU-bound tasks in parallel, thereby improving application performance. -
5
Kreuzberg
Polyglot document intelligence framework
Kreuzberg is a flexible task orchestration and agent workflow platform designed to help developers build, coordinate, and scale intelligent agents or automation pipelines that interact with external services, runtime environments, and multi-step business workflows. It emphasizes modular design so that developers can define discrete tasks or “actions” and then compose them into complex flows where dependencies, parallel steps, and error handling are declaratively managed. This structure makes it simpler to build resilient automation pipelines that span multiple technologies or APIs without needing to glue everything together manually. The platform also supports observability and monitoring, helping teams trace the progression of workflows, catch errors early, and evaluate performance trends in orchestrated systems. -
6
Bifrost
Middleware for production-oriented MySQL
...Multiple data sources, multiple target library support. Both incremental and full data synchronization are supported. One Binlog parsing thread, multiple target libraries are synchronized in parallel. Automatically restore configuration after restarting after downtime, accurate control of location. Support monitoring and alarm functions such as Email and WeChat enterprise account. Provide third-party monitoring platform access interface (Http Basic authentication) HTTP API. MySQL, ClickHouse plugin supports DDL synchronization. -
7
Colly
Elegant Scraper and Crawler Framework for Golang
...With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving. Clean API. Fast (>1k request/sec on a single core) Manages request delays and maximum concurrency per domain. Automatic cookie and session handling. Sync/async/parallel scraping. Distributed scraping. Caching, automatic encoding of non-unicode responses. Robots.txt support. Google App Engine support. -
8
future
R package: future: Unified Parallel and Distributed Processing in R
The future package in R provides a unified abstraction for asynchronous and/or parallel computation. It allows R expressions to be scheduled for future evaluation, with the result retrieved later, in a way decoupled from the specific backend used. This lets code be written in a way that works with sequential execution, multicore, multisession, cluster, or remote compute backends, without changing the high-level code. -
9
prek
Better `pre-commit`, re-engineered in Rust
...By rewriting everything in Rust, prek delivers substantial performance gains and reduces disk usage while maintaining compatibility with existing .pre-commit-config.yaml setups, letting teams switch over without rewriting configurations. It’s optimized for modern workflows, supporting monorepos with workspace modes, parallel hook installation and execution, and native handling of common hook tasks without external runtimes. Because it ships as a single binary with no runtime dependencies and is already being adopted in large projects (like CPython, Apache Airflow, and FastAPI), prek is increasingly seen as a next-generation tool for developers who value performance, simplicity, and robust git hook automation. -
10
OpenWorkflow
Open-source TypeScript framework for building durable workflows
...The framework supports long-running processes that can sleep for seconds or months, making it suitable for background jobs, automation pipelines, and event-driven systems. OpenWorkflow includes automatic retries with exponential backoff, parallel step execution, and scheduling capabilities, all while using the developer’s existing database instead of requiring dedicated orchestration infrastructure. A built-in dashboard provides visibility into workflow execution and debugging. Overall, it delivers production-ready workflow durability similar to temporal-style systems but with a simpler operational footprint. -
11
Pachyderm
Data-Centric Pipelines and Data Versioning
Data-driven pipelines automatically trigger based on detecting data changes. Automatic immutable data lineage and data versioning of all data types. Autoscaling and parallel processing built on Kubernetes for resource orchestration. Uses standard object stores for data storage with automatic deduplication. Runs across all major cloud providers and on-premises installations. Automatic and intelligent versioning of even the largest data sets of unstructured and structured data. Git-like structure enables effective team collaboration. ... -
12
Laravel Envoy
Elegant SSH tasks for PHP
Laravel Envoy is a tool for executing common tasks you run on your remote servers. Using Blade-style syntax, you can easily set up tasks for deployment, Artisan commands, and more. Currently, Envoy only supports the Mac and Linux operating systems. However, Windows support is achievable using WSL2. -
13
Buildbot
Python-based continuous integration testing framework
Buildbot is an open-source framework for automating software build, test, and release processes. At its core, Buildbot is a job scheduling system: it queues jobs, executes the jobs when the required resources are available, and reports the results. Your Buildbot installation has one or more masters and a collection of workers. The masters monitor source-code repositories for changes, coordinate the activities of the workers, and report results to users and developers. Workers run on a... -
14
CodeceptJS
Supercharged End 2 End Testing Framework for NodeJS
CodeceptJS is a modern end-to-end testing framework for Node.js applications, enabling developers to write tests in a behavior-driven development (BDD) style with an emphasis on simplicity and readability. -
15
ModelingToolkit.jl
Modeling framework for automatically parallelized scientific ML
ModelingToolkit.jl is a modeling language for high-performance symbolic-numeric computation in scientific computing and scientific machine learning. It then mixes ideas from symbolic computational algebra systems with causal and acausal equation-based modeling frameworks to give an extendable and parallel modeling system. It allows for users to give a high-level description of a model for symbolic preprocessing to analyze and enhance the model. Automatic symbolic transformations, such as index reduction of differential-algebraic equations, make it possible to solve equations that are impossible to solve with a purely numeric-based technique. ... -
16
Laravel Workflow
Durable workflow engine that allows users to track job status
Laravel Workflow is a package for the Laravel web framework that provides tools for defining and managing workflows and activities. A workflow is a series of interconnected activities that are executed in a specific order to achieve a desired result. Activities are individual tasks or pieces of logic that are executed as part of a workflow. Laravel Workflow can be used to automate and manage complex processes, such as financial transactions, data analysis, data pipelines, micro-services, job... -
17
S5CMD
Parallel S3 and local filesystem execution tool
s5cmd is a very fast S3 and local filesystem execution tool. It comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files. There are already other utilities to work with S3 and similar object storage services, thus it is natural to wonder what s5cmd has to offer that others don't. s5cmd supports wide range of object management tasks both for cloud... -
18
Orleans
Orleans is a cross-platform framework for building distributed apps
...It was created by Microsoft Research and introduced the Virtual Actor Model as a novel approach to building a new generation of distributed systems for the Cloud era. The core contribution of Orleans is its programming model which tames the complexity inherent to highly-parallel distributed systems without restricting capabilities or imposing onerous constraints. -
19
gradle-completion
Gradle tab completion for bash and zsh
Bash and Zsh completion support for Gradle. This provides fast tab completion for: Gradle tasks for the current project and sub-projects. Gradle CLI switches (e.g. --parallel). Common Gradle properties (e.g. -Dorg.gradle.debug) It also handles custom default build files, so rootProject.buildFileName = 'build.gradle.kts' is supported. See instructions for bash or for zsh, then consider optional additional configuration. Download and place the plugin and completion script into your oh-my-zsh plugins directory. ... -
20
mgmt
Next generation distributed, event-driven, parallel config management!
Mgmt is a real-time automation tool. It is familiar to existing configuration management software but is drastically more powerful as it can allow you to build real-time, closed-loop feedback systems, in a very safe way, and with a surprisingly small amount of our mcl code. For example, the following code will ensure that your file server is set to read-only when it's Friday. It can run continuously, intermittently, or on demand, and in the first case, it will guarantee that your system is... -
21
Deployer
Deployment tool with support for popular frameworks out of the box
A deployment tool written in PHP with support for popular frameworks out of the box. Deployer is a cli tool for deployment of any PHP applications, including frameworks such as Laravel, Symfony, Zend Framework and many more. Main concept of Deployer is recipe, a php file containing tasks definitions. Recipe can require other recipes and extend/ override functionality. Also Deployer comes with bunch of ready to use recipes from community for Slack, etc. Deployer can be easily installed via... -
22
Pester
Pester is the ubiquitous test and mock framework for PowerShell
...It integrates with CI systems easily—returning standard exit codes and generating NUnit/JUnit-style test reports—so PowerShell codebases can be validated in automated pipelines. The framework evolves with PowerShell itself, adding features for parallel execution, code coverage measurement, and test discovery to meet production needs. For teams, Pester encourages test-driven development and makes PowerShell deliverables more maintainable and trustworthy. -
23
Ginkgo
BDD testing framework for Go
Ginkgo is a Go testing framework built to help you efficiently write expressive and comprehensive tests using Behavior-Driven Development (“BDD”) style. It is best paired with the Gomega matcher library but is designed to be matcher-agnostic. These docs are written assuming you’ll be using Gomega with Ginkgo. They also assume you know your way around Go and have a good mental model for how Go organizes packages under $GOPATH. Ginkgo is tested against Go v1.6 and newer To install Go, follow... -
24
Ray
A unified framework for scalable computing
Modern workloads like deep learning and hyperparameter tuning are compute-intensive and require distributed or parallel execution. Ray makes it effortless to parallelize single machine code — go from a single CPU to multi-core, multi-GPU or multi-node with minimal code changes. Accelerate your PyTorch and Tensorflow workload with a more resource-efficient and flexible distributed execution framework powered by Ray. Accelerate your hyperparameter search workloads with Ray Tune. ... -
25
Argo Workflows
Workflow engine for Kubernetes
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition). Define workflows where each step in the workflow is a container. Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG). Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes. ...
