[[PageOutline]]

= !RefactorErl Console Interface =

When running !RefactorErl, you can control the tool via function calls typed into the Erlang shell. !RefactorErl has many features, and is quite a complex system with dozen of functions whose name we do not expect our users to remember. In order to ease and unify the command-level access to all the analysis and refactoring functionality, we have designed a group of Erlang functions that cover the most frequently used features of the tool. The module these functions are located in is called {{{ri}}}, you can use this module to interact easily with the tool. You can add files/directories to the database, run [[SemanticQuery|semantic queries]], create backups, or even do [[RefactoringSteps|transformations]] via this Erlang-level interface.

== Command-line help ==

Help can be acquired in {{{ri}}} with

{{{
#!erlang
ri:help().
}}}

or even shorter as

{{{
#!erlang
ri:h().
}}}

This function lists several topics, on which further help is available by

{{{
#!erlang
ri:h(Topic).
}}}

If you need specific help with a function, simply call it with an {{{_h}}} postfix to the name. For example, help for the function add is available by calling

{{{
#!erlang
ri:add_h()
}}}


== Compiling the tool ==

The tool can be compiled/recompiled by invoking

{{{
#!erlang
ri:build().
}}}

You can also specify build parameters, but this feature is mostly applied through the development, so please find the module documentation for the details. Note that this build function tries to compile the [[NifDB|NIF graph representation]] as well, if these have not been compiled yet. If you want to prevent this, you can use the {{{no_nif}}} build parameter.

== Managing files ==
See [wiki:ManagingFiles Managing your files and applications].

== Using transformations ==
Transformations can be called using their abbreviated
names, and the list of required parameters. These commands are listed in
[[RefactoringSteps|refactoring functionalities]].
There is another way to call a transormation. This way let the user to
choose: user wants to specify all of arguments or not. There are lots of cases
when the user can not specify all of the required arguments. In this case the
tool can help the user with interactions. The tool ask questions and the user
has to answer it to specify the missing arguments. The interactions also work if
there are problems with the given arguments.

== Manipulating the graph ==
You can reset the database by invoking

{{{
#!erlang
ri:reset().
}}}

This will remove all loaded files. This function should be called if the graph
gets corrupted.
You can add a checkpoint and can create a backup using

{{{
#!erlang
ri:backup().
}}}

If a previous backup is needed to be load, you can load it using

{{{
#!erlang
ri:restore(NeededBackup).
}}}

If the transformations you have performed are not satisfactory, you can go
back to the previous checkpoint using

{{{
#!erlang
ri:undo().
}}}

Notice, that restore only modify the contents of the database, but undo
modifies the contents of the files on the disk, too.
 
If the files, which had been loaded to the database, has been changed on the disk,
then you can reload them to the  database by calling

{{{
#!erlang
ri:database_synchronization().
}}}

You can initiate the database synchronization from the start up script, too. 
See [[StartUp| Starting the tool]] for further details.


The [[PosModes| positioning mode]] of the database can be converted by calling

{{{
#!erlang
ri:db_convert(ToPos).
}}}

where !ToPos is one of the following:\\
* '''abs''': converting database to absolute positioning mode.\\
* '''rel''': converting database to relative positioning mode.\\


== Inspecting the graph ==
You can draw the semantic representation graph of !RefactorErl by calling

{{{
#!erlang
ri:graph().
}}}

This function produces a .dot file (by default, graph.dot, although this can be
customised), which can be transformed to several visual formats using Graphviz.
One of these transformations is available from !RefactorErl for convenience:

{{{
#!erlang
ri:svg().
}}}

The representation can be altered:

{{{
#!erlang
ri:svg(OutFile, Filter).
}}}

where Filter is one of the following:\\
* '''all''': default, all edges except environmental ones are shown.\\
* '''syn''': only syntactic edges are shown.\\
* '''sem''': only semantic edges are shown.\\
* '''lex''': only lexical edges are shown.\\
* '''all_env''': all edges are shown, no filtering.\\
* '''ctx''': context related edges are shown.\\
* '''not_lex''': all edges except lexical ones are shown.\\
* '''dataflow''': dataflow related edges are shown.\\
* '''a list of the above''': shows the union of the designated subgraphs.\\

== Using queries ==
Queries can be invoked by either

{{{
#!erlang
ri:q(Query).
}}}

or

{{{
#!erlang
ri:q(Module, Regexp, Query).
}}}

The former is applicable when a query starts generally, such as

{{{
#!erlang
ri:q("mods.funs.name").
}}}

For those queries that begin from a selected position (these queries start
with "@" when used from Emacs), the second variant is required. As the console
cannot mark a position, the first and the second component indicate the starting
point for the query. The following example shows how to get all the variables
used in the body of the function '''f/2''' from the module '''m'''.

{{{
#!erlang
ri:q(m, "f\\(X, Y\\)", "@fun.var").
}}}

Additional options can be given to a [[SemanticQuery|semantic query]] in a proplist as the last
argument. The following arguments are currently recognized:\\
* '''{out,!FileName}''': write the textual output of a query to a file. If the Filename contains relative path, then it will be converted absolute by using the path of the current working directory as base.\\
* '''linenum''': prepends match sites with file and line number information.\\
similar to '''grep -n'''.
The following example outputs all defined functions with line numbers to a
file named result.txt and located in the current working directory.

{{{
#!erlang
ri:q("mods.funs",[linenum,{out,"result.txt"}]).
}}}

or

{{{
#!erlang
ri:q(m, "f\\(X, Y\\)", "@fun.var", [linenum,{out,"result.txt"}]).
}}}

There is a [[SemanticQuery|semantic queries]] page, where you can learn more about this topic.

Semantic queries may take for a long time. A list of the currently running queries can be queried using 

{{{
#!erlang
ri:get_running_queries().
}}}

A running query can be aborted by calling 

{{{
#!erlang
ri:kill_query(QueryID).
}}}

== Using metric queries ==
There is a [[MetricQuery|metric queries]] page, where you can read a general description about metric queries.

Please, note that all of the metrics can be used as a property in our semantic query language.

A metric query can be run by executing the followings:

{{{#!erlang
MetricQueryString = "show number_of_fun for module ('a','b')",
ri:metric(MetricQueryString).
}}}

=== Querying bad smells ===

On the [[MetricQuery|metric queries]] page, more detailed information can be found about bad smells, and the usage of the metric analyzer mode.

!RefactorErl console supports querying bad smells in the code. 

In order to utilize it, the metric analyzer mode had to be turned on first:

{{{#!erlang
ri:metricmode(on).
}}}

After the metric analyzer mode has been turned on, the bad smells can be queried:

{{{#!erlang
ri:metricmode(show).
}}}

which returns a term that describes the modules and functions where the metrics have values outside the user defined limits.

The metric analyzer mode can be turned off by:
{{{#!erlang
ri:metricmode(off).
}}}

== Analysis ==

=== Dependency analysis ===

There is a [[/wiki/Dependency|Dependency Analysis]] page, where you can learn more about this topic. 

The command-line interface offers three interface functions, which are:

1. For drawing:
{{{#!erlang
ri:draw_dep/1
}}}
2. For printing the result to stdout:
{{{#!erlang
ri:print_dep/1
}}}

3. For drawing smart graph:
{{{#!erlang
ri:generate_smart_graph/1
}}}


==== Options ====

The parameter of the interface functions is a proplist setting the options of the analysis. For the available options and parameters, see:
{{{#!erlang
ri:draw_dep_h/0
}}}
{{{#!erlang
ri:print_dep_h/0
}}}

The parameter of the smart graph generation is a proplist setting the options of the generation and of the analysis. For the available options and parameters, see:
{{{#!erlang
ri:generate_smart_graph_h/0
}}}

In the options you can specify entities with their name as atoms (only modules) and strings (functions as "Mod:fun/arity"), or with regular expressions.

{{{#!comment
=== Dependency analysis on function or module level ===
There is a [[/wiki/Dependency/Functions|Module and Function Dependencies]] page, where you can learn more about this topic. 

The command-line interface offers two interface functions, which are:

1. For drawing:
{{{#!erlang
ri:draw_dep/1
}}}
2. For printing the result to stdout:
{{{#!erlang
ri:print_dep/1
}}}

3. For drawing smart graph:
{{{#!erlang
ri:generate_smart_graph/1
}}}


==== Options ====

The parameter of the interface functions is a proplist setting the options of the analysis. The available options are:

 * {{{level (mod | func)}}}
    The level of the dependency query (module or function).
 * {{{type (dep | cycles | all)}}}
    Whether the investigation should be done on the whole graph ({{{dep}}}), or just on the cyclic part ({{{cycles}}}) (if exists). When printing out the result type {{{all}}} returns graph nodes, while {{{cycles}}} and {{{dep}}} returns the module or function names.
 * {{{otp (true | false)}}}
    Whether Erlang/OTP standard modules should be included in the analysis or not.
 * {{{gnode}}}
    List of entity or entities that should be the starting point of the analysis. Especially at function level, the list is compulsory when the functions are identified by their module, name, arity.
 * exception
    List of entities excluded from the analysis.
 * leaves
    List of those entities which should be included in the analysis, but their children should not (and consequently the children become exceptions).
 * {{{dot}}}
    The file path of the generated {{{.dot}}} graph description. Unless it is a non-existing absolute path, the graph will be placed into the {{{./dep_files}}} directory. This option is only available when using {{{draw_dep}}}.


The parameter of the smart graph generation is a proplist setting the options of the generation and of the analysis. The available options are:
 * {{{dependency_level (mod | func) }}}
    The level of the dependency examination (module or function).
 * {{{output_path ( string() ) }}}
    Smart graphs are stored in {{{.html}}} files. If this option is not set, the name of the output file will be randomly generated, and the {{{.html}}} file will be placed in your {{{tool/data}}} directory.
 * {{{dependency_options ( level, type, otp, exception, leaves ) }}}
    The possible options are the same as above. (See dependency analysis.)
For example:
{{{#!erlang
DepOpts = [{level, mod}, {type, all}, {gnode, example_mod}, {exception, []}, {leaves, []}, {otp, false}],
ri:generate_smart_graph([{dependency_level, mod}, {output_path, "/tmp/example_smart_graph.html"}, {dependency_options, DepOpts}]).
}}}


You can specify entities either with their identifier or with graph nodes (such as {{{{'$gn', func, 123}}}}). Modules can be specified with their names as atoms (e.g. 'mnesia'), while functions are specified by their MFA descriptor as a string (e.g. "io:format/2")

==== Examples for listing results ====

* Checking for cycles in module level.
{{{#!erlang
ri:print_dep([{level, mod}, {type, all}]).
}}}
  

* Checking for cycles in function level, and printing out names of the functions (Module:Function/Arity).
{{{#!erlang
ri:print_dep([{level, func}, {type, cycles}]).
}}}
{{{#!erlang
  [['foo:fv4/1','foo:fv4/1'],
  ['test3:p/1','test:fv6/1','test3:p/1'],
  ['cycle4:f4/1','cycle3:f3/1','cycle4:f4/1'],
  ['cycle2:fv2/1','cycle1:fv1/0','cycle2:fv2/1'],
  ['test:fv5/1','test:fv4/2','test:fv5/1'],
  ['cycle4:f5/1','cycle3:f6/1','cycle4:f5/1']]
}}}


* Checking for cycles in function level, and printing out the graph nodes of the functions.
{{{#!erlang
ri:print_dep([{level, func}, {type, all}]).
}}}
{{{#!erlang
  {"6 cycle(s)",
  {[[{'$gn',func,28},{'$gn',func,28}], 
  [{'$gn',func,29},{'$gn',func,37},{'$gn',func,29}],
  [{'$gn',func,7},{'$gn',func,9},{'$gn',func,7}],
  [{'$gn',func,2},{'$gn',func,1},{'$gn',func,2}],
  [{'$gn',func,36},{'$gn',func,35},{'$gn',func,36}],
  [{'$gn',func,8},{'$gn',func,6},{'$gn',func,8}]]}
}}}

* Checking for cycles in module level from a given node
{{{#!erlang
 ri:print_dep([{level, mod}, {gnode, [f1]}]). 
}}}
{{{#!erlang
  {true,[[{'$gn',module,24},
          {'$gn',module,25},
          {'$gn',module,24}]]}
}}}

* Checking for cycles in function level from a node given with its identifier
{{{#!erlang
ri:print_dep([{level, func}, {gnode, ["cycle4:f5/1"]}]).
}}}

* Checking the dependencies in module level of a node given with its identifier
{{{#!erlang
ri:print_dep([{level, mod}, {type, dep}, {gnode, [f1]}]).
}}}
{{{#!erlang
  [{"f1",["f3","f2"]}]
}}}


* Drawing a smart graph by giving the output path.
{{{#!erlang
ri:generate_smart_graph([{dependency_options, [{type, all}]}, {dependency_level, func}, {output_path, "/tmp/alma.html"}]).
}}}
{{{#!erlang
"Open a browser and go to file:///tmp/alma.html to see the result."
}}}

* Drawing a smart graph without the output path parameter.
{{{#!erlang
ri:generate_smart_graph([{dependency_options, [{type, all}]}, {dependency_level, mod}]).
}}}
{{{#!erlang
"Open a browser and go to file:///home/aszabo/Dokumentumok/refactorerl/trunk/tool/data/sq_g3IAA2QAFXJlZmFjdG9yZXJsQGxvY2FsaG9zdAMAABJVAAAAAAAAAAA.html to see the result."
}}}


=== Function block dependencies ===
In large systems, sets of applications (which themselves consist of several modules) are organised into bigger units; keeping in line with Ericsson terminology, we shall call these function blocks. We also seek dependencies between them, which is conceptually similar to dependencies between modules: a function block FB1 is dependent on a function block FB2 if a module from FB1 is dependent on one from FB2. This examination is also available from the command-line interface. You can read about the usage and about the topic on [[/wiki/Dependency/FunctionBlocks|Function blocks]] page. 

}}}

=== Logical layers analysis ===
In large program systems, groups of compilation units (in the case of Erlang, modules) usually form logical layers. A desired property of such systems is that code in one layer should only use the layer immediately below it, and conversely, provide functionality only for the layer immediately above it. If you would like to check whether a system observes this rule, you should visit the [[InterfaceLayers|Interface Layers]] page, which show you how to check it.

=== Duplicated code analysis ===
In large program systems often occure duplicated code, which is a computer programming term for a sequence of source code that occurs more than once. There are two ways in which two code sequences can be duplicates of each other: syntactically and functionally. This new feature can detect the syntactically similar duplicates. You can learn more about this topic and about the usage on CloneIdentifiErl page, whilst the services provided by this interface are detailed below.
==== Functions ====
* clone_identifierl/0: uses the default values of properties
* clone_identifierl/1: takes a proplist as described above

Regardless of the chosen algorithm all output format, which are illustrated below, can be requested.
* prettyprint: Returns the textual representation of the clones.
Command: {{{ri:clone_identifierl([{algorithm,sw_metrics}, {files,[ucl_alg_dm]}, {format,prettyprint}]).}}}
{{{
-------------------------------------------------------------------------------- 
Clone element: (found in ucl_alg_dm):
q_gen(QueryETS) ->
    fun(F, Ps) ->
        [{F, Ps1}] = ets:lookup(QueryETS, F),
        length(list_sort_intersect(Ps1, Ps))
    end.

-------------------------------------------------------------------------------- 
-------------------------------------------------------------------------------- 
Clone element: (found in ucl_alg_dm):
u_gen(UseETS) ->
    fun(P, Fs) ->
        [{P, Fs1}] = ets:lookup(UseETS, P),
        length(list_sort_intersect(Fs1, Fs))
    end.
-------------------------------------------------------------------------------- 
}}}

* file_and_loc: Returns file and position information related to the clones. When using this output format it is possible to set the position type.
The {{{linecol}}} position type returns the row and column where the found clone starts and ends. The scalar position type returns the character position where the clone starts and ends.

Command:[[BR]]
{{{ri:clone_identifierl([{algorithm,sw_metrics}, {files,[ucl_alg_dm]}, {format,file_and_loc}, {postype,linecol]).}}}


{{{
[[[{filepath,"/home/laptop/refactorerl/src/ucl_alg_dm.erl"},
   {startpos,{12,1}},
   {endpos,{16,8}}],
  [{filepath,"/home/laptop/refactorerl/src/ucl_alg_dm.erl"},
   {startpos,{41,1}},
   {endpos,{45,8}}]]]
}}}

Command: [[BR]]
{{{ri:clone_identifierl([{algorithm,sw_metrics}, {files,[ucl_alg_dm]}, {format,file_and_loc}, {postype,scalar]).}}}

{{{
[[[{filepath,"/home/laptop/refactorerl/src/ucl_alg_dm.erl"},
   {startpos,{47}},
   {endpos,{121}}],
  [{filepath,"/home/laptop/refactorerl/src/ucl_alg_dm.erl"},
   {startpos,{520}},
   {endpos,{576}}]]]
}}}

* nodes: Returns the internal identifiers of the found clone groups. It is a good choice if you wish to further process the result by scripting using the ris interface.
Command: {{{ri:clone_identifierl([{algorithm,sw_metrics}, {files,[ucl_alg_dm]}, {format,nodes}]).}}}
{{{
[[[{'$gn',form,2}],[{'$gn',form,3}]]]
}}}


==== Result query functions ====
||=Function=||=Parameters=||=Description=||=Example=||
|| stored_dupcode_results/0 || - || Returns the list of all saved results (name and information about the parameters of the analysis) || ri:stored_dupcode_results() ||
|| show_dupcode/1 || Name::atom() - Name associated with the result ||Displays the selected result using the default output format. || ri:show_dupcode(mydups) ||
|| show_dupcode/2 || Name::atom() - name associated with the result, Format::atom() - requested output format || Displays the selected result using the requested format. || ri:show_dupcode(mysearch,file_and_loc) ||
|| show_dupcode_group/2 || Name::atom() - name associated with the result, GroupNumber::integer() - the required ID of the clone group belonging to the requested result || Displays the given clone group of the result using the default output format. || ri:show_dupcode_group(mysearch,2) ||
|| show_dupcode_group/3 || Name::atom() - name associated with the result, GroupNumber::integer() - the required ID of the clone group belonging to the requested result, Format::atom() - requested output format || Displays the given clone group of the result using the default output format using the requested format. || ri:show_dupcode_group(mysearch,2,file_and_loc) ||
|| save_dupcode_result/2 || Name::atom() - name associated with the result, FilePath::string() - absolute file path || Saves the result to !FilePath using the default output format. || ri:save_dupcode_result(mysearch,"/home/laptop/Desktop/result.txt")
|| save_dupcode_result/3 || Name::atom() - name associated with the result, FilePath::string() - absolute file path, Format::atom() - requested output format || Saves the result to !FilePath using the specified output format. || ri:save_dupcode_result(mysearch,"/home/laptop/Desktop/result.txt", file_and_loc) ||


====  Exemplars  ==== 
In this section, we show some illustrative examples to get familiar with this feature.
* Simpliest case.
{{{#!erlang
ri:clone_identifierl().
}}}
* We are looking for all of the clones can be found in the database. We have much memory and a wide interest in any detectable clones, and we also allow a greater deviation of clones.
{{{#!erlang
ri:clone_identifierl([{algorithm, matrix}, {caching, true}, {max_invalid_seq_length, 3}, {diff_limit, 0.2}]).
}}}

* We want to find the duplicates of a specific, newly introduced library function (lib_module:new_fun/2).
{{{#!erlang
ri:clone_identifierl([{func_list, [{lib_module, new_fun, 2}]}]).
}}}

* We want to find either the whole function or even its subsequences as duplicates of a specific, newly introduced library function (lib_module:new_fun/2).
{{{#!erlang
ri:clone_identifierl([{algorithm, matrix}, {func_list, [{lib_module, new_fun, 2}]}]).
}}}

* We want to find the duplicates of every function located in a library module (lib_module).
{{{#!erlang
ri:clone_identifierl([{files, [lib_module]}]).
}}}

* We want to find either the whole function or even its subsequences as duplicates of every function located in a library module (lib_module).
{{{#!erlang
ri:clone_identifierl([{algorithm, matrix},{files, [lib_module]}]).
}}}

* We want to find fully syntactically similar clones only in the dups and lib_module module.
{{{#!erlang
ri:clone_identifierl([{algorithm, suffix_tree},{files, [dups,lib_module]}]).
}}}

* We want to find long, fully syntactically similar clones that are straightforward to be eliminated.
{{{#!erlang
ri:clone_identifierl([{algorithm, filtered_suffix_tree},{minlen,80}]).
}}}


=== Clustering ===
You can learn more about this topic on [[wiki:ModuleFunctionClustering]] page.

You can use the clustering algorithm from {{{ri}}} by calling the {{{ri:cluster/0}}} function. You have to choose between agglomerative and genetic clustering at first and then between function and module clustering. Based on your choice {{{ri}}} will ask all of the parameters necessary for the clustering.

Consider the following example:
{{{
ri:cluster().

Please choose an item from the list (blank to abort).
 
Please select an algorithm for clustering:
 1. Agglomerative
 2. Genetic
 type the index of your choice: 1 
Please choose an item from the list (blank to abort).
 
Please select an entity type for clustering:
 1. function
 2. module
 type the index of your choice: 2
Please answer the following questions (blank to abort).
 Module clustering with Agglomerative algorithm
 Modules to skip(Type: none for default) none
 Functions to skip(Type: none for default) none
 Transform function(Select: [none,zero_one]) zero_one
 Distance function(Select: [call_sum,weight]) weight
 Antigravity(Type: 0.5 for default) 0.5
 Merge Function(Type: smart for default) smart
 Save results to database: (y/n) -> n
}}}

The result is:
{{{
See the direct information feed below:
 Clustering results:
 [[erl_syntax_lib,erl_syntax,igor,erl_tidy,epp_dodger,erl_recomment,
  erl_prettypr,prettypr,erl_comment_scan]]
 [[erl_syntax_lib,erl_syntax,igor,erl_tidy,epp_dodger,erl_recomment,
  erl_prettypr,prettypr],
 ...
 Fitness Numbers:
 [1.0,0.9473684210526315,0.918918918918919,0.8823529411764706,
 0.6896551724137931,0.5384615384615384,0.45,0.25]
}}}


== Server management command list ==

Here's the list of supported server management commands:\\
{{{
* '''system_info()''': Returns information about the !RefactorErl System. \\
* '''add(FDML)''': add a module, file, directory or a list of these to the database. \\
* '''drop(FDML)''': drop a module from the database. \\
* '''ls()''': list files that are in the database. \\
* '''backup()''': update the backup (checkpoint). \\
* '''restore(!Backup)''': restore the given backup. \\
* '''undo()''': undo the transformation (rollback, only one step). \\
* '''clean()''': clean backups (delete all checkpoints). \\
* '''reset()''': reset the database to an empty state, but valid schema. \\
* '''database_synchronization()''': Synchronize the contents of the database with the contents of the disc. \\
* '''graph(Target)''': assume no options and call one of the next two. \\
* '''graph(Atom,Options)''': assume ".dot" extension and call the one below. \\
* '''graph(File,Options)''': draw the graph with the given options. \\
* '''svg()''': draw the graph to graph.svg and call Graphviz. \\
* '''svg(File)''' \\
* '''svg(File, Options)''' \\
The additional/modied commands, that you can use, if you use the [[NifDB|NIF database engine]]:\\
* '''backup()''': creates a backup. \\
* '''backup(!CommitLog)''': creates a backup as '''ri:backup/0''', but here the user can attach a commit log to the backup file. \\
* '''ls_backups()''': returns a lists of backups, that has been created before with '''ri:backup/0''' or '''ri:backup/1'''. \\
* '''backup_info(Backup)''': returns information about the given backup. \\
* '''create_graph(Name)''': creates a graph with the given name. \\
* '''rename_graph(!OldName, !NewName)''': renames a graph that has the given !OldName, with the given !NewName. \\
* '''ls_graphs()''': returns a list of the created graphs. \\
* '''actual_graph()''': returns the actual graph's name. \\
* '''load_graph(Name)''': loads the given graph. \\
* '''delete_graph(Name)''': removes the given graph. \\
* '''delete_all_graphs()''': removes all graphs. \\