# Next (Version determined when release is cut)

# 0.8.1

# New Features

- Tests can now provide response time constraints to indicate a test failure.
- New `bypass-cert-verification` configuration option to bypass ssl cert verification.

# Changes

- We've migrated to Hyper 1.0 and to Rustls.

# 0.8.0

Jikken now supports secrets, body schemas, and data generation to allow complex schema validation and fuzz testing.
This release opens up support for new types of testing, with more advanced features for data validation and generation.
As always, check the [docs](https://www.jikken.io/docs/) for more details.

# New Features

- Variables can now generate data based on provided constraints for improved fuzz testing
- Added support for variable embedding in `setup`, `clean-up`, and multi-stage response bodies
- Added support for secrets as a secure alternative to global variables
- New `bodySchema` field, which allows more descriptive response validation
- New `platformId` field, to assist in tracking test versions over time (as part of our SaaS platform offering)
- Body comparison now supports strict and non-strict modes
- Improved error messaging around invalid JSON bodies
- OpenApi test generation now supports ref links
- OpenApi test generation can now produce variable definitions for generating test data
- Stages in multi-stage tests can now have names via the `name` field
- New `NOW` and `NOW_UTC` built-in global variables, which contain the current timestamp in local and UTC time
- New `format` command which auto-formats your test files
- New `validate` command for basic test file validation, which provides a mechanism to generate Platform IDs

# Changes

- Displays an error and skips test if test definition contains unknown fields
- Displays a warning when user specified config file is not found
- Some example test API URIs have changed, and the associated tests have been updated
- Split definition for iterative array variables to use a new `valueSet` keyword
- VSCode extension now recognizes the `project` and `valueSet` keywords
- VSCode extension has better support for embedded variables
- Telemetry now includes test stage names
- Telemetry now receives skipped tests information
- Telemetry now receives projects and environments on session create to support alert filtering
- Telemetry now leverages the `platformId` field for tracking test versions over time
- Telemetry no longer sends global variables attached to test data, but instead sends them as part of the session
- Telemetry no longer uses the ID field, which is now reserved for referencing tests as requires
- Test iterations now count as distinct test runs for telemetry data
- Updated Example tests to better showcase tag usage
- Added test execution runtimes to CLI output
- Added restrictions for variable names, which must now contain only alphanumeric characters, hyphens, and underscores

# Bug Fixes

- CLI output now properly flushes prior to test completion (for long-running tests)
- Each run of a test iteration now correctly counts as a distinct test
- Test Definitions no longer include global variables in telemetry, which fixes an issue that may result in inconsistent test ids

# 0.7.2

# New Features:

- Cookie support now also respects `Secure` mode.

# Bugfixes:

- Fixed compare endpoints not properly comparing bodies.

# 0.7.1

# New Features:

- State Variables (extracted from responses) are now able to be embedded into Post Bodies and URLs for subsequent stage/test runs.
- Basic Cookie support. Cookies returned from API endpoints are now added to state and treated as Strict domain/path cookies which pass to future stages.

# 0.7.0

  This release marks a big milestone for Jikken. We've added support for JUnit output and the ability to generate
  tests based on OpenAPI Specs. While these functionalities are working, we plan to incrementally improve them
  over time based on feedback. We've also released a Jikken extension for VSCode to help with writing tests.

# New Features:

- Added a CLI argument which outputs test execution details in the common JUnit format.
- Added support for the `new` command which generates JKT files based on OpenAPI Spec files.
- Added a new command `jk list` which provides a helpful printout for discovered test files.
- Created a VSCode extension for our JKT definition files.
- Added support for variable embedding inside Request bodies.
- Added support for a `.jikkenignore` which lets you specify test files to ignore when executing the test runner.
- Added a `disable` field to the test definition format which allows you to skip tests which would otherwise be included.
- Added a `delay` field to the test definition format which allows you to pause after a test stage before continuing execution.
- Added a `description` field to the test definition format.

# Changes:

- Added local build scripts to support MacOS cross-compiling for Apple Silicon vs x86
- Auth headers are now automatically redacted when enabling test execution telemetry.
  Previously if you used variables for the auth header they were already being restricted but now even if you hardcode
  a secret/key/token in the header it will also be redacted upon telemetry transmission. We still recommend you use
  variables instead of hardcoding credentials in test files.
- Added a badge which tracks our Homebrew release version in the github repo.
- Improved messaging around which tests are executing, being skipped, and why.
- Improved default messaging so that errors/details print after initial test pass/fail line.
- The CLI tool now provides different exit codes based on test success/failure.

# 0.6.2

# New Features:

- Enable project and environment support for test runs. These can be defined via envvar, config, or test definitions.
- Added support for HTTP DELETE test definitions.
- Added CLI argument to provide non-standard config file locations.
- Added new global variable TODAY_UTC to provide today's date based on UTC as opposed to local time.
- Added an optional `name` field for test stages.

# Changes:

- Changed the field name for variable definitions from `dataType` to just `type`.
- The `type` value for Variable definitions now supports other cases (String, string, STRING... etc).
- The windows installer now associates `jkt` files with Jikken.

# Bugfixes:

- Fixed a race condition that would sometimes incorrectly report sessions passing when individual tests failed.
- Fixed a bug with auto-updating version comparisons. Under some conditions the tool would improperly report an older version as being newer.
- Fixed calls which have body data to properly calculate content-type and content-length headers.
- Fixed console colors for legacy windows terminals (ansi-support).

# 0.6.1

# Changes:

- Added unit tests for Config system.

# Bugfixes:

- A println! used for local testing was incorrectly merged into the 0.6.0 release. It has been removed.

# 0.6.0

**BREAKING CHANGE**
In agreement with user feedback, we've decided to make our first breaking change to the test format.
Variables are no longer injected based on the `$var$` convention. They now follow similar
patterns to JavaScript and Bash: `${var}`. Based on discussions and usage we felt it best
to make this change before the number of tests in the wild using variables grows too large.
As always we aim to minimize breaking changes, and since usage is increasing, in the future we'll
likely support backwards compatibility or automated tooling to migrate tests.

We don't foresee any additional breaking changes on the horizon.

# New Features:

- Glob support for matching test files.
- Variables now support loading data from files.
- Our website is now public, which includes a new and improved docs page. Lots of documentation is on the way.

# Changes:

- Adjusted cargo compiler flags for release, greatly reduces release binary size.
- Update cleanup stage definition to use "always" for the always executing request.
- Reduce excessive use of cloning.
- Jikken no longer scans for test files recursively by default. You can now accomplish this via GLOB or with the `-r` CLI argument.
- Update help contents printed to the console.
- We've changed the format for VARIABLE injection. You now must follow the `${var}` pattern instead of the `$var$` pattern.
- Additional unit tests and some code clean-up/refactoring. More to come.
- Added support for HTTP Verbs to be case insensitive (all lower, all upper, or capitalized).

# Bugfixes:

- If a test is not checking response bodies, the test will no longer fail if the response body is not valid JSON.
- If test runs are configured to exit early on failure, the telemetry session completion and console status messages now properly trigger.

# 0.5.0

# New Features:

- Add (optional, opt-in) telemetry support for the Jikken.IO webapp
- Add basic test file validation
- Add support for user-path based configuration file

# Changes:

- Refactored codebase for cleaner more idiomatic rust (module layout)
- Made config files truly optional. If you provide an incorrect config file you will now receive an error but the tests will still run. Previously it would exit early and refuse to run your tests.

# Bugfixes:

- Ignore and Add Parameter definitions now properly work for compare requests

# 0.4.0

# New Features:

- Update Jikken CLI to have command driven execution. Instead of `jk` automatically running tests there are now various commands `jk run`, `jk dryrun` etc.
- Add `jk new` command to create a test file that shows all possible definition options.
- Add `environment` flag to test definition, cli args, config, and environment variables.
- Add `--quiet` flag to disable console output.
- Add `--trace` flag for more detailed output, intended use is for Jikken developers

# Changes:

- Adjusted console output messages around various verbosity levels
- Updated CLI help descriptions

# Bugfixes:

- Fix cleanup stage to properly handle onsuccess and onfailure definitions

# 0.3.0

# New Features:

- Minor adjustments to Example Tests
- Clean-up dry-run console output
- Added support for variable injection into request body definitions
- Added checking for latest version and a self-update command
- Added support for staged test definitions
- Added support for test setup
- Added support for test cleanup
- Added basic example of a multistage test
- Created Windows Installer for releases

# 0.2.0

# New Features:

- Improved test execution with invalid urls. It now properly prints out helpful errors and fails the test
- Consolidated two modes of test runs into a single code path. This simplifies expanding test_runner functionality in the future
- Added dry-run mode. This is a new CLI argument (-d) which prints a description of the steps that will happen without actually calling the apis

# Bugfixes:

- Fix label when printing tests by number. They now print starting at 1 instead of 0.

# 0.1.0

  Initial release of the Jikken CLI tool.

# New Features:

- basic yaml/json test file format to define api tests
- configuration file to support `continue on failure` mode and global variables
- environment variables overriding config file and global variable definition
- tests can be tagged and execution can be invoked to limit based on provided tags
- http requests (GET, POST, PUT, PATCH)
- define headers and body data
- compare result body to pre-defined
- compare status code to pre-defined
- compare two http endpoint results against each other (body and status)
- variable embeddings into tests
- variable extraction from test responses to be embedded into later run tests
- test dependencies (to force order of running tests)
- ignore/prune parts of a json response when comparing body responses
- variable types include `int`, `string`, `date`, and `datetime`
- variables can define sequences for test iterations to cycle through
- date variables support modifiers of adding/subtracting days, weeks, or months