Install

To install RefactorErl you can use the referl script located in the bin directory of the release. Unzip the release and type the following command:

bin/referl -build tool

System requirements: Install page

Build configuration options: ​Parameters of the referl script

Starting the tool

To start the tool with the default config use: bin/referl. This will store the source code representation in a Mnesia database. However, to decrease memory footprint and speed up the tool you may want to use the Kyoto Cabinet backend of RefactorErl:

bin/referl -db kcmini

For further options please check the StartUp page

Interfaces: ri, web

RefactorErl? provides several ​user interfaces.

The ​interactive Erlang shell interface -- ri -- gives you a simple command (function call) based usage. To ask help about the parameters of the function you can use the helper functions, such as ri:h() or ri:*_h().

If you successfully installed Yaws to your machine, you can use the ​web interface of the tool as well. When you compile the tool you have to provide the path to the YAWS' ebin directory: bin/referl -build tool -yaws_path path_to_yaws_ebin

When starting the web interface with ri:start_web2/1, you have to ​configure the webserver. Without configuration, the webserver will be started at localhost:8001

Building a database

For a first time user, we suggest using the ri interface to build the database of RefactorErl from the source code. There are several options to analyse the source files. For details see the file management page. Here we discuss some basic scenarios.

You can add files using the ri:add/* functions. To add a single module just provide the path to the file as an argument: ri:add("path_to_file")

The same command can be used to add directories recursievely: ri:add("path_to_dir")

If your included files are located in a separate directory, please add them as an include environment> ri:addenv(include, "path_to_include_dir")

If your software follows the Erlang application hierarchy and you have a library of your software.

• Add the path to the application libarary as an environment variable: ri:addenv(appbase, "path_to_my_lib")
• Add the files using a subkey of the application base path and the name of the application> ri:add(my_lib, my_application)
• This mode helps RefactorErl to find the include files in the appropriate include folders.

Using the tool

RefactorErl, as stated in its name, originally started as a refactoring project for Erlang. During the years of development, the focus of the project was shifted to the direction of code comprehension and software maintenance support. The main features of the tool are:

• Refactoring
• Semantic queries
• Dependency detection and visualisation
• Software complexity metrics
• Bad smell detection
• Checking design rules
• Vulnerability detection
• Clustering
• Duplicated code detection
• etc

Demo

In the next examples we will use the source of the Mnesaia database as a target software.

• starting the tool: bin/referl -db kcmini
• building the database:

  Eshell V10.6.1  (abort with ^G)
  (refactorerl@localhost)1> ri:ls().
  {{ok,[]},{error,[]}}
  ok
  (refactorerl@localhost)2> ri:envs().
  output = original
  appbase = "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib"
  ok
  (refactorerl@localhost)3> ri:add(erlang, mnesia).
  Adding: /usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src
  | 5.20 kB/s >>>>>>>>>>>>>>>>>>>| [ 420/ 420] mnesia.erl
  | 6.78 kB/s >>>>>>>>>>>>>>>>>>>| [   5/   5] mnesia_app.erl
  | 8.12 kB/s >>>>>>>>>>>>>>>>>>>| [   3/   3] mnesia_backend_type.erl
  | 5.13 kB/s >>>>>>>>>>>>>>>>>>>| [  12/  12] mnesia_backup.erl
  |>>>>>>>>>>>>         6.05 kB/s| [  48/  87] mnesia_bup.erl
  ....
  (refactorerl@localhost)4> ri:ls().
  {{ok,["/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_app.erl",
        "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_controller.erl",
        "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_ext_sup.erl",
        "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_frag_hash.erl",
  ....
  

ri:ls() checks the content of the database, so it lists the already analysed files. ri:envs() checks the environmental variables of RefactorErl. You might realise that the path of the Erlang installation library was already set, so we can use its subkey to add the mnesia application: ri:add(erlang, mnesia).

The query ri:q("mods.loc:sum"). returns the line of code analyzed:

(refactorerl@localhost)5> ri:q("mods.loc:sum").
    sum = 24299

Using the query language

The semantic query language of RefactorErl can be used to gather information about the source code according to the interest of the developer.

The query language was designed according to the syntactic/semantic entities of the Erlang language. So it introduces files, macros, modules, functions, expressions, records, record fields, etc.

The detailed description of the queries can be found here. The description of the entities, its properties and selectors are defined ​here, but you can use the ? selector in ri: ri:q("mods.?"). For further examples please check this ​page.

Once you build a query, you need an initial selector to start. That can be either a position based entity selection @fun -- the function pointed in the web interface, or a global starting point, like mods -- all analysed modules.

Once you selected an entity, you may ask some property of that entity:

• mods.name -- the name of the module

or ask its connected entities:

• mods.funs -- functions defined in the modules

You can also filter the entities:

• mods[name=foo].funs.calls -- what are the functions that are called in the functions of the foo module

In the following, we will show some queries on the previously built database from the source of Mnesia.

Detecting relations, gathering information about the source code

• ri:q(mods[name=mnesia_log].funs). -- listing all the functions from the mnesia_log module
• ri:q(mods[name=mnesia_log].funs.name). -- listing the name of the functions from the mnesia_log module
• ri:q(mods[name=mnesia_log].funs.refs). -- listing the references (the function applications) of the the functions from the mnesia_log module
• ri:q(mods[name=mnesia_log].funs[.refs]).
• mods[name=mnesia_log].funs[name=open_log]
• mods[name=mnesia_log].funs[name=open_log, arity=4]
• mods[name=mnesia_log].funs[name=open_log].refs
• mods[name=mnesia_log].funs[name=open_log].called_by
• @fun.called_by
• mods[name=mnesia_log].funs[name=open_log].calls
• @fun.calls

• mods.records[name=mnesia_select]
• files.records[name=mnesia_select]

• mods.records[name=mnesia_select].refs
• @record.refs

• mods.records[name=mnesia_select]
• .field[name=orig].refs
• @field.refs

• mods.funs.exprs.sub[type=atom, value=mnesia_tid_locks]

• mods.funs.exprs.sub[type=string, value~"Error message.*"]

• mods[name=mnesia_log].funs.exprs.sub[type=tuple,[.sub[index=1].origin[type=atom,value =backup_args]]]

• mods[name=mnesia_log].funs[name=open_log].refs

• @expr.origin

• ri:anal_dyn()
• mods.funs.dynrefs

• mods[name=mnesia].funs[name=write, arity=1].dynrefs
• @fun.dynrefs

• mods[name=mnesia_log].funs[name=open_log].refs[.param[index=1].origin[type=atom, value=decision_log]]

• mods[name=mnesia].funs[name=foldr].called_by

• mods[name=mnesia].funs[name=foldr].called_by[mod /= mnesia]

• mods.macros
• mods.macro[name="DEBUG_TAB"].refs
• @macro.refs
• @expr.macro_value

Checking design rules

• mods.funs.is_tail_rec -- list whether a function is tail recursive
• mods.funs[loc>50] -- lists long functions
• mods.funs[max_depth_of_cases>3] -- list the functions that are too deeply nested
• mods.funs[branches_of_recursion>5] -- list those functions that considered to complax and has more than 5 recursive branches
• mods[max_length_of_line>80] -- lists the modules containing lines longer than 80 characters

Detecting vulnerabilities

• mods.unsecure_calls -- Lists all the possible vulnerabilities
  • mods.unsecure_interoperability -- Lists interoperability related weaknesses
  • mods.unsecure_concurrency -- Identifies concurrency related issues
  • mods.unsecure_os_call -- Checks for OS injection
  • mods.unsecure_port_creation -- Identifies port creation related issues
  • mods.unsecure_file_operation -- Lists unsecure file handling
  • mods.unstable_call -- Shows possible atom exhaustion
  • mods.nif_calls -- Identifies unsecure NIF calls
  • mods.unsecure_port_drivers -- Lists the unsecure ddll usage
  • mods.decommissioned_crypto -- Lists the legacy functions from crypto module
  • mods.unsecure_compile_operations -- Shows unsecure compile/code loading related operations
  • mods.unsecure_process_linkage -- Lists unsecure process linkage
  • mods.unsecure_prioritization -- Identifies unsecure process prioritization
  • mods.unsecure_ets_traversal -- Lists unsecure ETS traversal
  • mods.unsafe_network -- Checks for unsecure kernel related operation
  • mods.unsecure_xml_usage -- Identifies unsecure xml parsing
  • mods.unsecure_communication -- Lists unsecure communication related settings