```refactor-options hidden revert diff-style = full no-show-filename no-collapse-diff ``` Many refactoring commands in `c2rust refactor` are designed to work only on selected portions of the crate, rather than affecting the entire crate uniformly. To support this, `c2rust refactor` has a *mark* system, which allows marking AST nodes (such as functions, expressions, or type annotations) with simple string labels. Certain commands add or remove marks, while others check the existing marks to identify nodes to transform. For example, in a program containing several byte string literals, you can use `select` to mark a specific one: ```rust refactor-target hidden static B1: &'static [u8] = b"123"; static B2: &'static [u8] = b"abc"; static B3: &'static [u8] = b"!!!"; ``` ```refactor no-revert select target 'item(B2); desc(expr);' ``` Then, you can use `bytestr_to_str` to change only the marked byte string to an ordinary string literal, leaving the others unaffected: ```refactor bytestr_to_str ``` This ability to limit transformations to specific parts of the program is useful for refactoring a large codebase incrementally, on a module-by-module or function-by-function basis. The remainder of this tutorial describes `select` and related mark-manipulation commands. For details of how marks affect various transformation commands, see the [command documentation](commands.md) or read about the [`marked!` pattern](rewrite.md#marked) for `rewrite_expr` and other pattern-matching commands. # Marks A "mark" is a short string label that is associated with a node in the AST. Marks can be applied to nodes of most kinds, including items, expressions, patterns, type annotations, and so on. The mark string can be any valid Rust identifier, though most commands that process marks use short words such as `target`, `dest`, or `new`. It's possible to apply multiple distinct marks to the same node, and it's also possible to mark children of marked nodes separately from their parents (for example, to mark an expression and one of its subexpressions). Here are some examples. ```rust refactor-target hidden fn f() -> Option { Some(2 + 2) } fn g() -> i32 { match f() { Some(x) => x, None => 0, } } ``` ```refactor select target 'crate; desc(match_expr(2 + 2));' ``` The ▶ ... ◀ indicators in the diff show that the expression `2 + 2` has been marked. Hover over the indicators for more details, such as the label of the added mark. As mentioned above, most kinds of nodes can be marked, not only expressions. Here we mark a function, a pattern, and a type annotation: ```refactor select a 'item(f);' ; select b 'item(g); desc(match_ty(i32));' ; select c 'item(g); desc(match_pat(Some(x)));' ; ``` As mentioned above, it's possible to mark the same node twice with different labels. (Marking it twice with the same label is no different from marking it once.) Here's an example of marking a function multiple times: ```refactor select a 'item(f);' ; select a 'item(f);' ; select b 'item(f);' ; ``` As you can see by hovering over the indicators, labels `a` and `b` were both added to the function `f`. Marks on a node have no connection to marks on its parent or child nodes. We can, for example, mark an expression like `2 + 2`, then separately mark its subexpressions with either the same or different labels: ```refactor select a 'item(f); desc(match_expr(2 + 2));' ; select a 'item(f); desc(match_expr(2)); first;' ; select b 'item(f); desc(match_expr(2)); last;' ; ``` Hovering over the mark indicators shows precisely what has happened: we marked both `2 + 2` and the first `2` with the label `a`, and marked the second `2` with the label `b`. # The `select` command The `select` command provides a simple scripting language for applying marks to specific nodes. The basic syntax of the command is: ```sh select LABEL SCRIPT ``` `select` runs a `SCRIPT` (written in the language described below) to obtain a set of AST nodes, then marks every node in the set with `LABEL`, which should be a single identifier such as `target`. More concretely, when running the script, `select` maintains a "current selection", which is a set of AST nodes. Script operations (described below) can extend or modify the current selection. At the end of the script, `select` marks every node in the current selection with `LABEL`. We next describe a few common select script patterns, followed by details on the available operations and filters. ## Common patterns ### Selecting an item by path For items such as functions, type declarations, or traits, the `item(path)` operation selects the item by its path: ```rust refactor-target hidden fn f() {} trait T {} struct S {} mod m { fn g() {} } ``` ```refactor select target 'item(f);' ; select target 'item(T);' ; select target 'item(S);' ; select target 'item(m::g);' ; ``` Note that this only works for the kinds of items that can be imported via `use`. It doesn't handle other kinds of item-like nodes, such as impl methods, which cannot be imported directly. ### Selecting all nodes matching a filter The operations `crate; desc(filter);` together select all nodes (or, equivalently, all descendants of the crate) that match a filter. For example, we can select all expressions matching the pattern `2 + 2` using a `match_expr` filter: ```rust refactor-target hidden fn f() -> i32 { 2 + 2 } const FOUR: i32 = 2 + 2; static ARRAY: [u8; 2 + 2] = [1, 2, 3, 4]; ``` ```refactor select target 'crate; desc(match_expr(2 + 2));' ``` Here we see that `crate; desc(filter);` can find matching items anywhere in the crate: inside function bodies, constant declarations, and even inside the length expression of an array type annotation. ### Selecting filtered nodes inside a parent node In the previous example, `crate; desc(filter);` is made up of two separate script operations. `crate` selects the entire crate: ```refactor no-revert select target 'crate;' ``` Then `desc(filter)` looks for descendants of selected nodes that match `filter`, and replaces the current selection with the nodes it finds: ```refactor clear_marks ; select target 'crate; desc(match_expr(2 + 2));' ``` (Note: we use `clear_marks` here only for illustration purposes, to make the diff clearly show the changes between the old and new versions of our `select` command.) Combining `desc` with operations other than `crate` allows selecting descendants of only specific nodes. For example, we can find expressions matching `2 + 2`, but only within the function `f`: ```refactor select target 'item(f); desc(match_expr(2 + 2));' ``` In a more complex example, we can use multiple `desc` calls to target an expression inside of a specific method (recall that methods can't be selected directly with `item(path)`). We first select the module containing the impl: ```rust refactor-target hidden fn f() -> i32 { 2 + 2 } mod m { struct S; impl S { fn f(&self) -> i32 { 2 + 2 } } } ``` ```refactor no-revert select target 'item(m);' ``` Then we select the method of interest, using the `name` filter (described below): ```refactor no-revert clear_marks ; select target 'item(m); desc(name("f"));' ``` And finally, we select the expression inside the method: ```refactor clear_marks ; select target 'item(m); desc(name("f")); desc(match_expr(2 + 2));' ``` Combined with some additional filters described below, this approach is quite effective for marking nodes that can't be named with an ordinary import path, such as impl methods or items nested inside functions. ## Script operations A `select` script can consist of any number of operations, which will be run in order to completion. (There is no control flow in `select` scripts.) Each operation ends with a semicolon, much like Rust statements. The remainder of this section documents each script operation. ### `crate` `crate` (which takes no arguments) adds the root node of the entire crate to the current selection. All functions, modules, and other declarations are descendants of this single root node. Example: ```rust refactor-target hidden fn f() -> i32 { 123 } mod m { static S: i32 = 0; } ``` ```refactor select target 'crate;' ``` ### `item` `item(p)` adds the item identified by the path `p` to the current selection. The provided path is handled like in Rust's `use` declarations (except that only plain paths are supported, not wildcards or curly-braced blocks). ```rust refactor-target hidden fn f() -> i32 { 123 } mod m { static S: i32 = 0; } ``` ```refactor select target 'item(m::S);' ``` Because the `item` operation only adds to the current selection (as opposed to replacing the current selection with a set containing only the identified item), we can run `item` multiple times to select several different items at once: ```refactor select target 'item(f); item(m::S); item(m);' ``` ### `child` `child(f)` checks each child of each currently selected node against the filter `f`, and replaces the current selection with the set of matching children. This can be used, for example, to select a `static`'s type annotation without selecting type annotations that appear inside its initializer: ```rust refactor-target hidden static S: i32 = 123_u8 as i32; const C: u32 = 0; ``` ```refactor select target 'item(S); child(kind(ty));' ``` To illustrate how this works, here is the AST for the `static S` item: * item `static S` * identifier `S` (the name of the `static`) * type `i32` (the type annotation of the `static`) * expression `123_u8 as i32` (the initializer of the `static`) * expression `123_u8` (the input of the cast expression) * type `i32` (the target type of the cast expression) The `static`'s type annotation is a direct child of the static (and has kind `ty`, matching the `kind(ty)` filter), so the type annotation is selected by the example command above. The target type for the cast is not a direct child of the static - rather, it's a child of the initializer expression, which is a child of the static - so it is ignored. ### `desc` `desc(f)` ("descendant") checks each descendant of each currently selected node against the filter `f`, and replaces the current selection with the set of matching descendants. This is similar to `child`, but checks for matching descendants at any depth, not only matching direct children. Using the same example as for `child`, we see that `desc` selects more nodes: ```refactor select target 'item(S); desc(kind(ty));' ``` Specifically, it selects both the type annotation of the `static` and the target type of the cast expression, as both are descendants of the `static` (though at different depths). Of course, it still does not select the type annotation of the `const C`, which is not a descendant of `static S` at any depth. Note that `desc` only considers the *strict* descendants of marked nodes - that is, it does not consider a node to be a "depth-zero" descendant of itself. So, for example, the following command selects nothing: ```refactor select target 'item(S); desc(item_kind(static));' ``` `S` itself is a `static`, but contains no additional statics inside of it, and `desc` does not consider `S` itself when looking for `item_kind(static)` descendants. ### `filter` `filter(f)` checks each currently selected node against the filter `f`, and replaces the current selection with the set of matching nodes. Equivalently, `filter(f)` removes from the current selection any nodes that don't match `f`. Most uses of the `filter` operation can be replaced by passing a more appropriate filter expression to `desc` or `child`, so the examples in this section are somewhat contrived. (`filter` can still be useful in combination with `marked`, described below, or in more complex select scripts.) Here is a slightly roundabout way to select all items named `f`. First, we select all items: ```rust refactor-target hidden fn f() {} fn g() {} mod m { fn f() {} } ``` ```refactor no-revert select target 'crate; desc(kind(item));' ``` Then, we use `filter` to keep only items named `f`: ```refactor clear_marks ; select target 'crate; desc(kind(item)); filter(name("f"));' ``` With this command, only descendants of crate matching both filters `kind(item)` and `name("f")` are selected. (This could be written more simply as `crate; desc(kind(item) && name("f"));`.) ### `first` and `last` `first` replaces the current selection with a set containing only the first selected node. `last` does the same with the last selected node. "First" and "last" are determined by a postorder traversal of the AST, so sibling nodes are ordered as expected, and a parent node come "after" all of its children. The `first` and `last` operations are most useful for finding places to insert new nodes (such as with the [`create_item` command](commands.md#create_item)) while ignoring details such as the specific names or kinds of the nodes around the insertion point. For example, we can use `last` to easily select the last item in a module. First, we select all the module's items: ```rust refactor-target hidden mod m { fn f() {} static S: i32 = 0; const C: i32 = 1; } ``` ```refactor no-revert select target 'item(m); child(kind(item));' ``` Then we use `last` to select only the last such child: ```refactor clear_marks ; select target 'item(m); child(kind(item)); last;' ``` Now we could use [`create_item`](commands.md#create_item) to insert a new item after the last existing one. ### `marked` `marked(l)` adds all nodes marked with label `l` to the current selection. This is useful for more complex marking operations, since (together with the `delete_marks` command) it allows using temporary marks to manipulate multiple sets of nodes simultaneously. For example, suppose we wish to select both the first and the last item in a module. Normally, this would require duplicating the `select` command, since both `first` and `last` replace the entire current selection with the single first or last item. This would be undesirable if the operations for setting up the initial set of items were fairly complex. But with `marked`, we can save the selection before running `first` and restore it afterward. ```rust refactor-target hidden mod m { fn f() {} static S: i32 = 0; const C: i32 = 1; } ``` We begin by selecting all items in the module and saving that selection by marking it with the `tmp_all_items` label: ```refactor no-revert select tmp_all_items 'item(m); child(kind(item));' ``` Next, we use `marked` to retrieve the `tmp_all_items` set and take the first item from it. This reduces the current selection to only a single item, but the `tmp_all_items` marks remain intact for later use. ```refactor no-revert select target 'marked(tmp_all_items); first;' ``` We do the same to mark the last item with `target`: ```refactor no-revert select target 'marked(tmp_all_items); last;' ``` Finally, we clean up, removing the `tmp_all_items` marks using the `delete_marks` command: ```refactor delete_marks tmp_all_items ``` Now the only marks remaining are the `target` marks on the first and last items of the module, as we originally intended. ### `reset` `reset` clears the set of marked nodes. This is only useful in combination with `mark` and `unmark`, as otherwise the operations before a `reset` have no effect. ### `mark` and `unmark` These operations allow `select` scripts to manipulate marks directly, rather than relying solely on the automatic marking of selected nodes at the end of the script. `mark(l)` marks all nodes in the current selection with label `l` (immediately, rather than waiting until the `select` command is finished), and `unmark(l)` removes label `l` from all selected nodes. `mark`, `unmark`, and `reset` can be used to effectively combine multiple `select` commands in a single script. Here's the "first and last" example from the `marked` section, using only a single `select` command: ```refactor select _dummy ' item(m); child(kind(item)); mark(tmp_all_items); reset; marked(tmp_all_items); first; mark(target); reset; marked(tmp_all_items); last; mark(target); reset; marked(tmp_all_items); unmark(tmp_all_items); reset; ' ``` Note that we pass `_dummy` as the `LABEL` argument of `select`, since the desired `target` marks are applied using the `mark` operation, rather than relying on the implicit marking done by `select`. `unmark` is also useful in combination with `marked` to interface with non-`select` mark manipulation commands. For example, suppose we want to mark all occurrences of `2 + 2` that are passed as arguments to a function `f`. One option is to do this using the `mark_arg_uses` command, with additional processing by `select` before and after. Here we start by marking the function `f`: ```rust refactor-target hidden fn f(x: i32) { // ... } fn g(x: i32) { // ... } fn main() { f(1); f(2 + 2); g(2 + 2); let x = 2 + 2; } ``` ```refactor no-revert select target 'item(f);' ``` Next, we run `mark_arg_uses` to replace the mark on `f` with a mark on each argument expression passed to `f`: ```refactor no-revert mark_arg_uses 0 target ``` And finally, we use `select` again to mark only those arguments that match `2 + 2`: ```refactor select target 'marked(target); unmark(target); filter(match_expr(2 + 2));' ``` Beginning the script with `marked(target); unmark(target);` copies the set of `target`-marked nodes into the current selection, then removes the existing marks. The remainder of the script can then operate as usual, manipulating only the current selection with no need to worry about additional marks being already present. ## Filters ### Boolean operators Filter expressions can be combined using the boolean operators `&&`, `||`, and `!`. A node matches the filter `f1 && f2` only if it matches `f1` and also matches `f2`, and so on. ### `kind` `kind(k)` matches AST nodes whose node kind is `k`. The supported node kinds are: * `item` - a top-level item, as in `struct Foo { ... }` or `fn foo() { ... }`. Includes both items in modules and items defined inside functions or other blocks, but does not include "item-like" nodes inside traits, impls, or `extern` blocks. * `trait_item` - an item inside a trait definition, such as a method or associated type declaration * `impl_item` - an item inside an impl block, such as a method or associated type definition * `foreign_item` - an item inside an `extern block` ("foreign module"), such as a C function or static declaration * `stmt` * `expr` * `pat` - a pattern, including single-ident patterns like `foo` in `let foo = ...;` * `ty` - a type annotation, such as `Foo` in `let x: Foo = ...;` * `arg` - a function or method argument declaration * `field` - a struct, enum variant, or union field declaration * `itemlike` - matches nodes whose kind is any of `item`, `trait_item`, `impl_item`, or `foreign_item` * `any` - matches any node The node kind `k` can be used alone as shorthand for `kind(k)`. For example, the operation `desc(item);` is the same as `desc(kind(item));`. ### `item_kind` `item_kind(k)` matches itemlike AST nodes whose subkind is `k`. The itemlike subkinds are: * `extern_crate` * `use` * `static` * `const` * `fn` * `mod` * `foreign_mod` * `global_asm` * `ty` - type alias definition, as in `type Foo = Bar;` * `existential` - existential type definition, as in `existential type Foo: Bar;`. Note that existential types are currently an unstable language feature. * `enum` * `struct` * `union` * `trait` - ordinary `trait Foo { ... }` definition, including `unsafe trait` * `trait_alias` - trait alias definition, as in `trait Foo = Bar;` Note that trait aliases are currently an unstable language feature. * `impl` - including both trait and inherent impls * `mac` - macro invocation. Note that `select` works on the macro-expanded AST, so macro invocations are never present under normal circumstances. * `macro_def` - 2.0/`decl_macro`-style macro definition, as in `macro foo(...) { ... }`. Note that 2.0-style macro definitions are currently an unstable language feature. Note that a single `item_kind` filter can match multiple distinct node kinds, as long as the subkind is correct. for example, `item_kind(fn)` will match `fn` `item`s, method `trait_item`s and `impl_item`s, and `fn` declarations inside `extern` blocks (`foreign_item`s). similarly, `item_kind(ty)` matches ordinary `type` alias definitions, associated type declarations (in traits) and definitions (in impls), and foreign type declarations inside `extern` blocks. `item_kind` filters match only those nodes that also match `kind(itemlike)`, as other node kinds have no itemlike subkind. The itemlike subkind `k` can be used alone as shorthand for `item_kind(k)`. For example, the operation `desc(fn);` is the same as `desc(item_kind(fn));`. ### `pub` and `mut` `pub` matches any item, impl item, or foreign item whose visibility is `pub`. It currently does not support struct fields, even though they can also be declared `pub`. `mut` matches `static mut` items, `static mut` foreign item declarations, and mutable binding patterns such as the `mut foo` in `let mut foo = ...;`. ### `name` `name(re)` matches itemlikes, arguments, and fields whose name matches the regular expression `re`. For example, `name("[fF].*")` matches `fn f() { ... }` and `struct Foo { ... }`, but not `trait Bar { ... }`. It currently does not support general binding patterns, aside from those in function arguments. ### `path` and `path_prefix` `path(p)` matches itemlikes and enum variants whose absolute path is `p`. `path_prefix(n, p)` is similar to `path(p)`, but drops the last `n` segments of the node's path before comparing to `p`. ### `has_attr` `has_attr(a)` matches itemlikes, exprs, and field declarations that have an attribute named `a`. ### `match_*` `match_expr(e)` uses [`rewrite_expr`-style AST matching](rewrite.md) to compare exprs to `e`, and matches any node where AST matching succeeds. For example, `match_expr(__e + 1)` matches the expressions `1 + 1`, `x + 1`, and `f() + 1`, but not `2 + 2`. `match_pat`, `match_ty`, and `match_stmt` are similar, but operate on pat, ty, and stmt nodes respectively. ### `marked` `marked(l)` matches nodes that are marked with the label `l`. ### `any_child`, `all_child`, `any_desc`, and `all_desc` `any_child(f)` matches nodes that have a child that matches `f`. `all_child(f)` matches nodes where all children of the node match `f`. `any_desc` and `all_desc` are similar, but consider all descendants instead of only direct children. ## Other commands In addition to `select`, `c2rust refactor` contains a number of other mark-manipulation commands. A few of these can be replicated with appropriate `select` scripts (though using the command is typically easier), but some are more complex. ### `copy_marks` `copy_marks OLD NEW` adds a mark with label `NEW` to every node currently marked with `OLD`. ### `delete_marks` `delete_marks OLD` removes the label `OLD` from every node that is currently marked with it. ### `rename_marks` `rename_marks OLD NEW` behaves like `copy_marks OLD NEW` followed by `delete_marks OLD`: it adds a mark with label `NEW` to every node marked with `OLD`, then removes `OLD` from each such node. ### `mark_uses` `mark_uses LABEL` transfers `LABEL` marks from definitions to uses. That is, it finds each definition marked with `LABEL`, marks each use of such a definition with `LABEL`, then removes `LABEL` from the definitions. For example, if a `static FOO: ... = ...` is marked with `target`, then `mark_uses target` will add a `target` mark to every expression `FOO` that references the marked definition and then remove `target` from `FOO` itself. For the purposes of this command, a "use" of a definition is a path or identifier that resolves to that definition. This includes expressions (both paths and struct literals), patterns (paths to constants, structs, and enum variants), and type annotations. When a function definition is marked, only the function path itself (the `foo::bar` in `foo::bar(x)`) is considered a use, not the entire call expression. Method calls (whether using dotted or UFCS syntax) normally can't be handled at all, as their resolution is "type-dependent" (however, the `mark_callers` command can sometimes work when `mark_uses` does not). ### `mark_callers` `mark_callers LABEL` transfers `LABEL` marks from function or method definitions to uses. That is, it works like `mark_uses`, but is specialized to functions and methods. `mark_callers` uses more a more sophisticated means of name resolution that allows it to detect uses via type-dependent method paths, which `mark_uses` cannot handle. For purposes of `mark_callers`, a "use" is a function call (`foo::bar()`) or method call (`x.foo()`) expression where the function or method being called is one of the marked definitons. ### `mark_arg_uses` `mark_arg_uses INDEX LABEL` transfers `LABEL` marks from function or method definitions to the argument in position `INDEX` at each use. That is, it works like `mark_callers`, but marks the expression passed as argument `INDEX` instead of the entire call site. `INDEX` is zero-based. However, the `self`/receiver argument of a method call counts as the first argument (index 0), with the first argument in parentheses having index 1 (`arg0.f(arg1, arg2)`). For ordinary function calls (including UFCS method calls), the first argument has index 0 (`f(arg0, arg1, arg2)`)