dyer

Overview

dyer is designed for reliable, flexible and fast Request-Response based service, including data processing, web-crawling and so on, providing some friendly, interoperable, comprehensive features without compromising speed.

Features

• asynchronous, concurrent streaming and I/O, make the best of thread pool, network, and system resource.
• Event-driven, once you set the initials and recursive generator, dyer will handle the rest of it interoperably.
• User-friendly and flexible, dyer offers high-level, flexible, easy to use wrappers and APIs what does a lot for you.

Feature Flag

To reduce code redundancy and speed up compilation, dyer use feature flag to mark the necessary modules/functions, Currently here are some supported Features:

• xpath: Enable parse the html response with xpath
• compression: Enable HTTP Compression: br, deflate, gzip
• proxy: Enable use proxies
• full: Enable all features

Guide

Get started by installing dyer-cli and looking over the examples.

The Cookbook gives a detailed view of dyer.