Crates.io | gis_puller |
lib.rs | gis_puller |
version | 0.1.1 |
source | src |
created_at | 2021-05-19 01:55:01.073229 |
updated_at | 2021-12-16 04:58:00.965685 |
description | Geographic Information System (GIS) datasets acquisition library. |
homepage | https://github.com/pinkforest/gis_puller |
repository | https://github.com/pinkforest/gis_puller |
max_upload_size | |
id | 399269 |
size | 118,572 |
This Rust crate aims to abstract and implement a general global Geographic Information System (GIS) data catalog and acquisition functionality.
Acquired catalog will follow the data provider derived structural separations as-is.
To build on top the below crates can be in future used to consume the acquired data and beyond:
Everything should be considered unstable including the interface definitions.
Also see DIFFICULTIES.md
Using Rust Cargo - requires cargo-edit
cargo add gis_puller
Or edit Cargo.toml by hand:
[dependencies]
gis_puller = "0.1.0"
Create documentation
env GIS_CONFIG_PATH=config RUN_MODE=development cargo doc --no-deps --open
Run tests (includes doctests and runs examples) with either replay or mocked datasources
env GIS_CONFIG_PATH=config TEST_DATA_PATH=test_data RUN_MODE=development cargo test
Run clippy for additional lints
cargo clippy --all-targets --all-features
Run tarpaulin for code/examples test coverage
env GIS_CONFIG_PATH=config TEST_DATA_PATH=test_data RUN_MODE=development cargo +nightly tarpaulin --run-types Tests,Doctests,Benchmarks,Examples,Lib,Bins -v
NOTE: Tarpaulin requires +nightly: rustup toolchain install nightly
@TODO binary crate -- To use in shell with gis_puller image Docker e.g. with Australia localities boundaries datasets obtained from data.gov.au:
docker run --rm -t -i -e env GIS_PULLER_CONFIGPATH=/gis/etc/gis_puller GIS_PULLER_DATAPATH=/gis/data gis_puller gis_puller pull --all au locality/boundary
Or to use as a library:
Currently with AU locality/boundary dataset I doctest fetcher_matcher the below in development the below with recorded result replayed as a mock:
mod mocks;
use mocks::http_server;
use httpmock::Method::GET;
use tokio::task;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let au_config = task::spawn_blocking(move || {
gis_puller::settings::Settings::load("au")
}).await?.unwrap();
// @TODO: replace this mess with a macro_rules!
let mut au_fetcher_matcher = (*au_config).fetcher_matcher.clone();
let mock_server = mocks::http_server::serve().await?;
let mock_path = format!("/replay/data.gov.au/api/v0/search/datasets");
let mock_query = format!("localities-boundaries");
let inject_replay = format!("test_data/replay/au/locality_boundaries.json");
let inject_url = format!("http://localhost:{}{}", mock_server.port(), &mock_path);
au_fetcher_matcher.rest_url = Some(inject_url);
au_fetcher_matcher.query = Some(mock_query.clone());
let mock = mock_server.mock(|when, then| {
when.method(GET)
.path(&mock_path)
.query_param("query", &mock_query);
then.status(200)
.header("Content-Type", "application/json")
.body_from_file(&inject_replay);
});
let catalog = gis_puller::au::fetcher_matcher(&au_fetcher_matcher).await?;
println!("Catalog = {:#?}", catalog);
Ok(())
}
Scratchpad for future interface:
let puller_au = gis_puller::Puller::load('au').await?;
let sources_au = puller_au.sources('locality/boundary').await?;
// make sources_au Iterable for .iter()
sources_au.pull_all('/tmp').await?;
sources_au.pull_one('nsw').await?;
The below features can be toggled via Cargo:
Licensed under either of:
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.