kql-language-tools

Rust bindings to Microsoft's Kusto.Language library for KQL (Kusto Query Language) validation, completion, and syntax classification.

Overview

This crate provides native Rust access to the official KQL parser and language services via .NET AOT compilation. It enables:

• Syntax Validation - Check KQL queries for syntax errors
• Schema Validation - Validate queries against a database schema (tables, columns, functions)
• Completions - Get intellisense suggestions at cursor position
• Classification - Get syntax highlighting spans for queries

Prerequisites

Required Dependencies by Platform

macOS

# Install .NET SDK via Homebrew
brew install dotnet

# Or download from Microsoft
# https://dotnet.microsoft.com/download/dotnet/8.0

# Xcode Command Line Tools (for C compiler)
xcode-select --install

Required:

• .NET 8.0+ SDK
• Xcode Command Line Tools (provides clang)

Linux (Ubuntu/Debian)

# Install .NET SDK
# Option 1: Package manager (Ubuntu 22.04+)
sudo apt-get update
sudo apt-get install -y dotnet-sdk-8.0

# Option 2: Microsoft packages
wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y dotnet-sdk-8.0

# Install build essentials (C compiler)
sudo apt-get install -y build-essential

Required:

• .NET 8.0+ SDK
• GCC or Clang (build-essential package)

Linux (Fedora/RHEL)

# Install .NET SDK
sudo dnf install dotnet-sdk-8.0

# Install C compiler
sudo dnf install gcc

Required:

• .NET 8.0+ SDK
• GCC

Windows

# Option 1: Download installer from Microsoft
# https://dotnet.microsoft.com/download/dotnet/8.0

# Option 2: Using winget
winget install Microsoft.DotNet.SDK.8

# Option 3: Using Chocolatey
choco install dotnet-sdk

Required:

• .NET 8.0+ SDK
• Visual Studio Build Tools or Visual Studio with C++ workload
  • Download from: https://visualstudio.microsoft.com/visual-cpp-build-tools/

Verifying Installation

# Check .NET SDK version
dotnet --version
# Should output 8.0.x or higher

# Check C compiler (macOS/Linux)
cc --version

# Check C compiler (Windows - in Developer Command Prompt)
cl

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Rust Application                         │
│                                                              │
│  KqlValidator::new()?                                        │
│    .validate_syntax("T | where x > 1")?                     │
│    .get_completions("T | ", 4, Some(&schema))?              │
│    .get_classifications("T | take 10")?                     │
└──────────────────────────┬──────────────────────────────────┘
                           │ C ABI
┌──────────────────────────▼──────────────────────────────────┐
│                  .NET Native Library                         │
│                  (KqlLanguageFfiNE.dylib/.so/.dll)          │
│                                                              │
│  Microsoft.Azure.Kusto.Language                              │
│  - KustoCode.ParseAndAnalyze()                               │
│  - CodeScript.GetCompletionItems()                           │
│  - Syntax tree classification                                │
└─────────────────────────────────────────────────────────────┘

Installation

Add to your Cargo.toml:

[dependencies]
kql-language-tools = "0.1"

Note: The native library is built automatically by cargo build if not present (requires .NET SDK and C compiler).

API Reference

KqlValidator

The main entry point for all language services.

use kql_language_tools::{KqlValidator, Schema, Table};

// Create a validator instance (loads native library)
let validator = KqlValidator::new()?;

Syntax Validation

Check a query for syntax errors without schema awareness:

let result = validator.validate_syntax("SecurityEvent | where TimeGenerated > ago(1h)")?;

if result.is_valid() {
    println!("Query is valid!");
} else {
    for diagnostic in result.diagnostics() {
        println!("{}:{}: {} - {}",
            diagnostic.line,
            diagnostic.column,
            diagnostic.severity,
            diagnostic.message
        );
    }
}

Schema Validation

Validate queries against a known schema:

let schema = Schema::new()
    .table(Table::new("SecurityEvent")
        .with_column("TimeGenerated", "datetime")
        .with_column("Account", "string")
        .with_column("Computer", "string"));

let result = validator.validate_with_schema(
    "SecurityEvent | project TimeGenerated, UnknownColumn",
    &schema
)?;

// Will report error: "UnknownColumn" not found
assert!(!result.is_valid());

Completions (Intellisense)

Get completion suggestions at a cursor position:

// Without schema - returns keywords and operators
let completions = validator.get_completions("SecurityEvent | ", 16, None)?;

for item in &completions.items {
    println!("{} ({:?})", item.label, item.kind);
    // "where (Keyword)", "project (Keyword)", "summarize (Keyword)", etc.
}

// With schema - includes table and column names
let completions = validator.get_completions(
    "SecurityEvent | project ",
    24,
    Some(&schema)
)?;
// Returns: TimeGenerated, Account, Computer, plus functions...

CompletionItem fields:

• label - Display text
• kind - Keyword, Function, Table, Column, etc.
• insert_text - Text to insert (if different from label)
• detail - Brief description or signature
• sort_order - Priority (lower = higher priority)
• edit_start - Character position where replacement starts

Classification (Syntax Highlighting)

Get classified spans for syntax highlighting:

let result = validator.get_classifications("SecurityEvent | where x > 1")?;

for span in &result.spans {
    println!("{}..{}: {:?}",
        span.start,
        span.start + span.length,
        span.kind
    );
}
// 0..13: Identifier (SecurityEvent)
// 14..15: QueryOperator (|)
// 16..21: QueryOperator (where)
// 22..23: Identifier (x)
// 24..25: ScalarOperator (>)
// 26..27: Literal (1)

ClassificationKind variants:

• PlainText, Comment, Punctuation, Directive
• Literal, StringLiteral, Type, Identifier
• Column, Table, Database, Cluster
• ScalarFunction, AggregateFunction
• Keyword, Operator, Variable, Parameter
• QueryOperator, ScalarOperator

Types

ValidationResult

pub struct ValidationResult {
    pub valid: bool,
    pub diagnostics: Vec<Diagnostic>,
}

impl ValidationResult {
    fn is_valid(&self) -> bool;
    fn has_errors(&self) -> bool;
    fn has_warnings(&self) -> bool;
    fn errors(&self) -> impl Iterator<Item = &Diagnostic>;
    fn warnings(&self) -> impl Iterator<Item = &Diagnostic>;
}

Diagnostic

pub struct Diagnostic {
    pub message: String,
    pub severity: DiagnosticSeverity,  // Error, Warning, Information, Hint
    pub start: usize,   // Character offset
    pub end: usize,
    pub line: usize,    // 1-based
    pub column: usize,  // 1-based
    pub code: Option<String>,
}

Schema

let schema = Schema::new()
    .with_database("MyDatabase")
    .table(Table::new("Events")
        .with_column("Timestamp", "datetime")
        .with_column("Message", "string")
        .with_column("Level", "int")
        .with_description("Application events"))
    .function(Function::new("GetRecentEvents")
        .with_parameter("hours", "int")
        .with_return_type("dynamic")
        .with_body("Events | where Timestamp > ago(hours * 1h)"));

Building the Native Library

Automatic Build (Recommended)

The native library is built automatically when you run cargo build:

cargo build

The build script will:

1. Check if the native library already exists
2. If not, detect if .NET SDK is available
3. Automatically run the build script to compile the native library
4. Provide helpful instructions if .NET SDK is not installed

Prerequisites

• .NET 8.0 SDK or later
• Platform-specific C compiler (Xcode on macOS, gcc on Linux, MSVC on Windows)

Manual Build (Optional)

If you prefer to build manually or need cross-platform builds:

cd dotnet

# Build for current platform
./build.sh

# Build for specific platform
./build.sh osx-arm64
./build.sh linux-x64

# Build all platforms (requires cross-compilation setup)
./build.sh all

Supported platforms:

• osx-arm64 - macOS Apple Silicon
• osx-x64 - macOS Intel
• linux-x64 - Linux x86_64
• linux-arm64 - Linux ARM64
• win-x64 - Windows x86_64
• win-arm64 - Windows ARM64

Output

The build produces:

dotnet/native/{rid}/
├── KqlLanguageFfiNE.dylib    # Native entry point (or .so/.dll)
├── KqlLanguageFfi.dll         # Managed assembly
├── Kusto.Language.dll         # Kusto parser
└── KqlLanguageFfi.runtimeconfig.json

Testing

Run Tests

The library auto-detects DOTNET_ROOT on most systems (including Homebrew installations):

# Run all tests (including integration tests that require native library)
cargo test -- --include-ignored

# Run with output visible
cargo test -- --include-ignored --nocapture

# Run specific test
cargo test test_get_completions -- --include-ignored

Manual DOTNET_ROOT (if auto-detection fails)

If tests fail with runtime errors, manually set DOTNET_ROOT:

# macOS with Homebrew
export DOTNET_ROOT=/opt/homebrew/Cellar/dotnet/9.0.8/libexec

# Or find your .NET installation
export DOTNET_ROOT=$(dirname $(dirname $(which dotnet)))

Test Coverage

Library Loading

The native library is searched in this order:

1. KQL_LANGUAGE_TOOLS_PATH environment variable (file or directory)
2. Same directory as the executable
3. dotnet/native/{rid}/ relative to the crate
4. Current working directory

To override:

export KQL_LANGUAGE_TOOLS_PATH=/path/to/native/osx-arm64

C API Contract

For consumers building their own bindings, the C ABI functions are:

// Lifecycle
int32_t kql_init(void);
void kql_cleanup(void);

// Validation
int32_t kql_validate_syntax(
    const uint8_t* query, int32_t query_len,
    uint8_t* output, int32_t output_max_len
);

int32_t kql_validate_with_schema(
    const uint8_t* query, int32_t query_len,
    const uint8_t* schema_json, int32_t schema_len,
    uint8_t* output, int32_t output_max_len
);

// Completions
int32_t kql_get_completions(
    const uint8_t* query, int32_t query_len,
    int32_t cursor_position,
    const uint8_t* schema_json, int32_t schema_len,  // nullable
    uint8_t* output, int32_t output_max_len
);

// Classification
int32_t kql_get_classifications(
    const uint8_t* query, int32_t query_len,
    uint8_t* output, int32_t output_max_len
);

// Error retrieval
int32_t kql_get_last_error(uint8_t* output, int32_t output_max_len);

Return codes:

• > 0 - Success, value is JSON length written to output
• 0 - Success, empty/valid result
• -1 - Buffer too small
• -2 - Parse error in input
• -3 - Internal error

Platform Support

License

MIT OR Apache-2.0

The Kusto.Language library is provided by Microsoft under its own license terms.