RefactorErl Console Interface

When running RefactorErl, you can control the tool via function calls typed into the Erlang shell. RefactorErl has many features, and is quite a complex system with dozen of functions whose name we do not expect our users to remember. In order to ease and unify the command-level access to all the analysis and refactoring functionality, we have designed a group of Erlang functions that cover the most frequently used features of the tool. The module these functions are located in is called ri, you can use this module to interact easily with the tool. You can add files/directories to the database, run semantic queries, create backups, or even do transformations via this Erlang-level interface.

Command-line help

Help can be acquired in ri with

ri:help().

or even shorter as

ri:h().

This function lists several topics, on which further help is available by

ri:h(Topic).

If you need specific help with a function, simply call it with an _h postfix to the name. For example, help for the function add is available by calling

ri:add_h()

Compiling the tool

The tool can be compiled/recompiled by invoking

ri:build().

You can also specify build parameters, but this feature is mostly applied through the development, so please find the module documentation for the details. Note that this build function tries to compile the NIF graph representation as well, if these have not been compiled yet. If you want to prevent this, you can use the no_nif build parameter.

Managing files

You can add files to the RefactorErl database by calling the add function with either a filename as a string or a module name as an atom. Note that in the latter case, "ri" defaults to the current working directory (which you may work around by including a path in your singe-quoted atom). If you specify a directory instead of a regular filename, then it will be recursively traversed. You may just as well give a list of atoms or strings to add more files at once. All of the following example commands would add the same file:

 cd(dir), ri:add(modname).
 ri:add('dir/modname').
 ri:add(['dir/modname']).
 ri:add("dir/modname.erl").
 ri:add("/current/dir/modname.erl").
 ri:add("path_to_dir/dir").

The module displays the progression of loading. Removing files from the database is similarly easy and also recursive. The following will equally work:

 ri:drop(modname).
 ri:drop([modname]).
 ri:drop("dir/modname.erl").
 ri:drop("/current/dir/modname.erl").
 ri:drop("path_to_dir/dir").

Modules can be loaded as applications, but the base of your library has to be set before:

ri:addenv(appbase, "path/to/my/applib").

If the application has additional include directories, then these directories has to be also set in advance:

ri:addenv(include, "path/to/my/incldir").

If the application build process contains compile-time macros, then these macros also can be set:

%if it is only used in ifdef forms.
ri:addenv(def, 'my_macro').

ri:addenv(def, {'my_macro', my_macro_value}).

If the include forms of the application contain OS based enviromental nodes, then these nodes can be set:

ri:addenv(env_var, {os_env_name, "os_env_path"}).

Let's see an example:

The Emakefile has the following contents:

{"/home/user/dev/lib/app1/src/*",
   [{i,"/home/user/dev/lib/"},
    {d,'MY_MACRO', "ITS_VALUE"},
    {outdir,"/home/user/dev/lib/app1/ebin"}]}.

{"/home/user/dev/lib/app2/src/*",
   [{i,"/home/user/dev/lib/"},
    {d,'MY_MACRO', "ITS_VALUE"},
    {outdir,"/home/user/dev/lib/app2/ebin"}]}.

{"/home/user/dev/lib/app3/src/*",
   [{i,"/home/user/dev/lib/share/include/"},
    {i,"/home/user/dev/lib/"},
    {outdir,"/home/user/dev/lib/app3/ebin"}]}.

To add app1, app2, app3 to RefactorErl you should do the followings:

% add app1, app2 with the same configuration
ri:addenv(appbase, "/home/user/dev/lib/"),
ri:addenv(def, {'MY_MACRO', "ITS_VALUE"}),

Apps = [app1, app2],
[ri:add(home, App) || App <- Apps].

% add app3 after the configuration has been modified
ri:delenv(def),
ri:addenv(include, "/home/user/dev/lib/share/include/"),

ri:add(home, app3).

You can check the already given application base directories by listing all of the enviromental nodes:

ri:envs().

It is possible to delete the defined environment variables:

ri:delenv(include).

Or you can set an environmental variable to another value:

ri:setenv(env_name, "path/to/new_value").

For convenience, both the filenames and the directory names can be given as atoms as well as strings. The list of loaded files can be obtained by calling

ri:ls().

This call also displays the status of the loaded files (error or no_error). If the module m is loaded,

ri:ls(m).

will give information about the functions, records and macros in the file. The contents of a file can be listed by

ri:cat(ModFile).

The content of a function can be listed by

ri:cat(ModFile, {FunName,Arity}).

Usually, Erlang source files (having the extension .erl) are loaded into RefactorErl. In addition, RefactorErl is also capable of loading compiled .beam files.

ri:add("compiled.beam").

Note that this feature is applicable only to those .beam files that were compiled with the debug_info option. Also note that the resulting file will be pretty printed by RefactorErl.

Using transformations

Transformations can be called using their abbreviated names, and the list of required parameters. These commands are listed in refactoring functionalities. There is another way to call a transormation. This way let the user to choose: user wants to specify all of arguments or not. There are lots of cases when the user can not specify all of the required arguments. In this case the tool can help the user with interactions. The tool ask questions and the user has to answer it to specify the missing arguments. The interactions also work if there are problems with the given arguments.

Manipulating the graph

You can reset the database by invoking

ri:reset().

This will remove all loaded files. This function should be called if the graph gets corrupted. You can add a checkpoint and can create a backup using

ri:backup().

If a previous backup is needed to be load, you can load it using

ri:restore(NeededBackup).

If the transformations you have performed are not satisfactory, you can go back to the previous checkpoint using

ri:undo().

Notice, that restore only modify the contents of the database, but undo modifies the contents of the files on the disk, too. If the files, which had been loaded to the database, has been changed on the disk, then you can reload them to the database by calling

ri:database_synchronization().

You can initiate the database synchronization from the start up script, too. See Starting the tool for further details.

Inspecting the graph

You can draw the semantic representation graph of RefactorErl by calling

ri:graph().

This function produces a .dot file (by default, graph.dot, although this can be customised), which can be transformed to several visual formats using Graphviz. One of these transformations is available from RefactorErl for convenience:

ri:svg().

The representation can be altered:

ri:svg(OutFile, Filter).

where Filter is one of the following:

• all: default, all edges except environmental ones are shown.
  
• syn: only syntactic edges are shown.
  
• sem: only semantic edges are shown.
  
• lex: only lexical edges are shown.
  
• all_env: all edges are shown, no filtering.
  
• ctx: context related edges are shown.
  
• not_lex: all edges except lexical ones are shown.
  
• dataflow: dataflow related edges are shown.
  
• a list of the above: shows the union of the designated subgraphs.
  

Using queries

Queries can be invoked by either

ri:q(Query).

or

ri:q(Module, Regexp, Query).

The former is applicable when a query starts generally, such as

ri:q("mods.funs.name").

For those queries that begin from a selected position (these queries start with "@" when used from Emacs), the second variant is required. As the console cannot mark a position, the first and the second component indicate the starting point for the query. The following example shows how to get all the variables used in the body of the function f/2 from the module m.

ri:q(m, "f\\(X, Y\\)", "@fun.var").

Additional options can be given to a semantic query in a proplist as the last argument. The following arguments are currently recognized:

• {out,FileName}: write the textual output of a query to a file.
  
• linenum: prepends match sites with file and line number information.
  

similar to grep -n. The following example outputs all defined functions with line numbers to a file named result.txt.

ri:q("mods.funs",[linenum,{out,"result.txt"}]).

There is a semantic queries page, where you can learn more about this topic.

Semantic queries may take for a long time. A list of the currently running queries can be queried using

ri:get_running_queries().

A running query can be aborted by calling

ri:kill_query(QueryID).

Analysis

Dependency analysis on function or module level

There is a Module and Function Dependencies page, where you can learn more about this topic.

The command-line interface offers two interface functions, which are:

1. For drawing:

   ri:draw_dep/1
   

2. For printing the result to stdout:

   ri:print_dep/1
   

Options

The parameter of the interface functions is a proplist setting the options of the analysis. The available options are:

• level (mod | func)

The level of the dependency query (module or function).

• type (all | cycles)

Whether the investigation should be done on the whole graph, or just on the cyclic part (if exists). When printing out the cycles, type all returns graph nodes, while cycles returns names.

• otp (true | false)

Whether Erlang/OTP standard modules should be included in the analysis or not.

• gnode

List of entity or entities that should be the starting point of the analysis. Especially at function level, the list is compulsory when the functions are identified by their module, name, arity.

• exception

List of entities excluded from the analysis.

• leaves

List of those entities which should be included in the analysis, but their children should not (and consequently the children become exceptions).

• dot

The file path of the generated .dot graph description. Unless it is a non-existing absolute path, the graph will be placed into the ./dep_files directory. This option is only available when using draw_dep.

You can specify entities either with graph nodes (such as {'$gn', func, 123}) or with their identifier. Modules can be specified with their names as atoms (e.g. 'mnesia'), while functions are specified by their MFA descriptor as a string (e.g. "io:format/2")

Examples for listing results

• Checking for cycles in module level.

  ri:print_dep([{level, mod}, {type, all}]).
  

• Checking for cycles in function level, and printing out names of the functions (Module:Function/Arity).

  ri:print_dep([{level, func}, {type, cycles}]).
  

    [['foo:fv4/1','foo:fv4/1'],
    ['test3:p/1','test:fv6/1','test3:p/1'],
    ['cycle4:f4/1','cycle3:f3/1','cycle4:f4/1'],
    ['cycle2:fv2/1','cycle1:fv1/0','cycle2:fv2/1'],
    ['test:fv5/1','test:fv4/2','test:fv5/1'],
    ['cycle4:f5/1','cycle3:f6/1','cycle4:f5/1']]
  

• Checking for cycles in function level, and printing out the graph nodes of the functions.

  ri:print_dep([{level, func}, {type, all}]).
  

    {"6 cycle(s)",
    {[[{'$gn',func,28},{'$gn',func,28}], 
    [{'$gn',func,29},{'$gn',func,37},{'$gn',func,29}],
    [{'$gn',func,7},{'$gn',func,9},{'$gn',func,7}],
    [{'$gn',func,2},{'$gn',func,1},{'$gn',func,2}],
    [{'$gn',func,36},{'$gn',func,35},{'$gn',func,36}],
    [{'$gn',func,8},{'$gn',func,6},{'$gn',func,8}]]}
  

• Checking for cycles in module level from a given node

  ri:print_dep([{level, mod}, {gnode, {'$gn', module, 24}}]). 
  

    {true,[[{'$gn',module,24},
            {'$gn',module,25},
            {'$gn',module,24}]]}
  

• Checking for cycles in function level from a node given with its identifier

  ri:print_dep([{level, func}, {gnode, ["cycle4:f5/1"]}]).
  

Function block dependencies

In large systems, sets of applications (which themselves consist of several modules) are organised into bigger units; keeping in line with Ericsson terminology, we shall call these function blocks. We also seek dependencies between them, which is conceptually similar to dependencies between modules: a function block FB1 is dependent on a function block FB2 if a module from FB1 is dependent on one from FB2. This examination is also available from the command-line interface. You can read about the usage and about the topic on Function blocks page.

Logical layers analysis

In large program systems, groups of compilation units (in the case of Erlang, modules) usually form logical layers. A desired property of such systems is that code in one layer should only use the layer immediately below it, and conversely, provide functionality only for the layer immediately above it. If you would like to check whether a system observes this rule, you should visit the Interface Layers page, which show you how to check it.

Duplicated code analysis

In large program systems often occure duplicated code, which is a computer programming term for a sequence of source code that occurs more than once. There are two ways in which two code sequences can be duplicates of each other: syntactically and functionally. This new feature can detect the syntactically similar duplicates. You can learn more about this topic and about the usage, especially about the command-line usage, on Duplicate code analysis? page.

Clustering

You can learn more about this topic on ModuleFunctionClustering page.

You can use the clustering algorithm from ri by calling the ri:cluster/0 function. You have to choose between agglomerative and genetic clustering at first and then between function and module clustering. Based on your choice ri will ask all of the parameters necessary for the clustering.

Consider the following example:

ri:cluster().

Please choose an item from the list (blank to abort).
 
Please select an algorithm for clustering:
 1. Agglomerative
 2. Genetic
 type the index of your choice: 1 
Please choose an item from the list (blank to abort).
 
Please select an entity type for clustering:
 1. function
 2. module
 type the index of your choice: 2
Please answer the following questions (blank to abort).
 Module clustering with Agglomerative algorithm
 Modules to skip(Type: none for default) none
 Functions to skip(Type: none for default) none
 Transform function(Select: [none,zero_one]) zero_one
 Distance function(Select: [call_sum,weight]) weight
 Antigravity(Type: 0.5 for default) 0.5
 Merge Function(Type: smart for default) smart
 Save results to database: (y/n) -> n

The result is:

See the direct information feed below:
 Clustering results:
 [[erl_syntax_lib,erl_syntax,igor,erl_tidy,epp_dodger,erl_recomment,
  erl_prettypr,prettypr,erl_comment_scan]]
 [[erl_syntax_lib,erl_syntax,igor,erl_tidy,epp_dodger,erl_recomment,
  erl_prettypr,prettypr],
 ...
 Fitness Numbers:
 [1.0,0.9473684210526315,0.918918918918919,0.8823529411764706,
 0.6896551724137931,0.5384615384615384,0.45,0.25]

Server management command list

Here's the list of supported server management commands:

• add(FDML): add a module, file, directory or a list of these to the database.
  
• drop(FDML): drop a module from the database.
  
• ls(): list files that are in the database.
  
• backup(): update the backup (checkpoint).
  
• restore(!Backup): restore the given backup.
  
• undo(): undo the transformation (rollback, only one step).
  
• clean(): clean backups (delete all checkpoints).
  
• reset(): reset the database to an empty state, but valid schema.
  
• database_synchronization(): Synchronize the contents of the database with the contents of the disc.
  
• graph(Target): assume no options and call one of the next two.
  
• graph(Atom,Options): assume ".dot" extension and call the one below.
  
• graph(File,Options): draw the graph with the given options.
  
• svg(): draw the graph to graph.svg and call Graphviz.
  
• svg(File)
  
• svg(File, Options)
  

The additional/modied commands, that you can use, if you use the NIF database engine:

• backup(): creates a backup.
  
• backup(CommitLog): creates a backup as ri:backup/0, but here the user can attach a commit log to the backup file.
  
• ls_backups(): returns a lists of backups, that has been created before with ri:backup/0 or ri:backup/1.
  
• backup_info(Backup): returns information about the given backup.
  
• create_graph(Name): creates a graph with the given name.
  
• rename_graph(OldName, NewName): renames a graph that has the given OldName, with the given NewName.
  
• ls_graphs(): returns a list of the created graphs.
  
• actual_graph(): returns the actual graph's name.
  
• load_graph(Name): loads the given graph.
  
• delete_graph(Name): removes the given graph.
  
• delete_all_graphs(): removes all graphs.