wiki:howto

Version 12 (modified by tothm, 4 years ago) (diff)

--

Install

To install RefactorErl you can use the referl script located in the bin directory of the release. Unzip the release and type the following command:

bin/referl -build tool

System requirements: Install page

Build configuration options: ​Parameters of the referl script

Starting the tool

To start the tool with the default config use: bin/referl. This will store the source code representation in a Mnesia database. However, to decrease memory footprint and speed up the tool you may want to use the Kyoto Cabinet backend of RefactorErl:

bin/referl -db kcmini

For further options please check the StartUp page

Interfaces: ri, web

RefactorErl? provides several ​user interfaces.

The ​interactive Erlang shell interface -- ri -- gives you a simple command (function call) based usage. To ask help about the parameters of the function you can use the helper functions, such as ri:h() or ri:*_h().

If you successfully installed Yaws to your machine, you can use the ​web interface of the tool as well. When you compile the tool you have to provide the path to the YAWS' ebin directory: bin/referl -build tool -yaws_path path_to_yaws_ebin

When starting the web interface with ri:start_web2/1, you have to ​configure the webserver. Without configuration, the webserver will be started at localhost:8001

Building a database

For a first time user, we suggest using the ri interface to build the database of RefactorErl from the source code. There are several options to analyse the source files. For details see the file management page. Here we discuss some basic scenarios.

You can add files using the ri:add/* functions. To add a single module just provide the path to the file as an argument: ri:add("path_to_file")

The same command can be used to add directories recursievely: ri:add("path_to_dir")

If your included files are located in a separate directory, please add them as an include environment> ri:addenv(include, "path_to_include_dir")

If your software follows the Erlang application hierarchy and you have a library of your software.

  • Add the path to the application libarary as an environment variable: ri:addenv(appbase, "path_to_my_lib")
  • Add the files using a subkey of the application base path and the name of the application> ri:add(my_lib, my_application)
  • This mode helps RefactorErl to find the include files in the appropriate include folders.

Using the tool

RefactorErl, as stated in its name, originally started as a refactoring project for Erlang. During the years of development, the focus of the project was shifted to the direction of code comprehension and software maintenance support. The main features of the tool are:

  • Refactoring
  • Semantic queries
  • Dependency detection and visualisation
  • Software complexity metrics
  • Bad smell detection
  • Checking design rules
  • Vulnerability detection
  • Clustering
  • Duplicated code detection
  • etc

Demo

In the next examples we will use the source of the Mnesaia database as a target software.

  • starting the tool: bin/referl -db kcmini
  • building the database: 
    Eshell V10.6.1  (abort with ^G)
(refactorerl@localhost)1> ri:ls().
{{ok,[]},{error,[]}}
ok
(refactorerl@localhost)2> ri:envs().
output = original
appbase = "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib"
ok
(refactorerl@localhost)3> ri:add(erlang, mnesia).
Adding: /usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src
| 5.20 kB/s >>>>>>>>>>>>>>>>>>>| [ 420/ 420] mnesia.erl
| 6.78 kB/s >>>>>>>>>>>>>>>>>>>| [   5/   5] mnesia_app.erl
| 8.12 kB/s >>>>>>>>>>>>>>>>>>>| [   3/   3] mnesia_backend_type.erl
| 5.13 kB/s >>>>>>>>>>>>>>>>>>>| [  12/  12] mnesia_backup.erl
|>>>>>>>>>>>>         6.05 kB/s| [  48/  87] mnesia_bup.erl
....
(refactorerl@localhost)4> ri:ls().
{{ok,["/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_app.erl",
      "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_controller.erl",
      "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_ext_sup.erl",
      "/usr/local/Cellar/erlang/22.2.1/lib/erlang/lib/mnesia-4.16.2/src/mnesia_frag_hash.erl",
....

ri:ls() checks the content of the database, so it lists the already analysed files. ri:envs() checks the environmental variables of RefactorErl. You might realise that the path of the Erlang installation library was already set, so we can use its subkey to add the mnesia application: ri:add(erlang, mnesia).

The query ri:q("mods.loc:sum"). returns the line of code analyzed: 

(refactorerl@localhost)5> ri:q("mods.loc:sum").
    sum = 24299

Using the query language

The semantic query language of RefactorErl can be used to gather information about the source code according to the interest of the developer.

The query language was designed according to the syntactic/semantic entities of the Erlang language. So it introduces files, macros, modules, functions, expressions, records, record fields, etc.

The detailed description of the queries can be found here. The description of the entities, its properties and selectors are defined ​here, but you can use the ? selector in ri: ri:q("mods.?"). For further examples please check this ​page.

Once you build a query, you need an initial selector to start. That can be either a position based entity selection @fun -- the function pointed in the web interface, or a global starting point, like mods -- all analysed modules.

Once you selected an entity, you may ask some property of that entity:

  • mods.name -- the name of the module

or ask its connected entities:

  • mods.funs -- functions defined in the modules

You can also filter the entities:

  • mods[name=foo].funs.calls -- what are the functions that are called in the functions of the foo module

In the following, we will show some queries on the previously built database from the source of Mnesia.

Detecting relations, gathering information about the source code

  • ri:q(mods[name=mnesia_log].funs). -- lists all the functions from the mnesia_log module
  • ri:q(mods[name=mnesia_log].funs.name). -- lists the name of the functions from the mnesia_log module
  • ri:q(mods[name=mnesia_log].funs.refs). -- lists the references (the function applications) of the the functions from the mnesia_log module
  • ri:q(mods[name=mnesia_log].funs[.refs]). -- lists the function which are called somewhere ([.ref] behaes like an embedded query. If the result is empty, that is a false filetr, otherwise it is true)
  • mods[name=mnesia_log].funs[name=open_log] -- search the definitions of the mnesio_log:open_log functions
  • mods[name=mnesia_log].funs[name=open_log, arity=4] -- search the definition of the mnesio_log:open_log/4 function
  • mods[name=mnesia_log].funs[name=open_log].refs -- search the references of the mnesio_log:open_log functions
  • mods[name=mnesia_log].funs[name=open_log].called_by -- search the functions that calls the mnesio_log:open_log function
  • @fun.called_by -- search the functions that call the pointed function
  • mods[name=mnesia_log].funs[name=open_log].calls -- search the functions that are called from the mnesio_log:open_log functions
  • @fun.calls -- search the functions tha call the pointed function
  • mods.records[name=mnesia_select] -- list the mnesia_select record definition
  • files.records[name=mnesia_select] -- list the mnesia_select record definiton
  • mods.records[name=mnesia_select].refs -- list the record usgaes of the mnesia_select record
  • @record.refs -- list the usages of the pointed record
  • mods.records[name=mnesia_select].field[name=orig].refs -- list the mnesia_select record expressions where the orig field is used
  • @field.refs -- list the references of the pointed record field
  • mods.funs.exprs.sub[type=atom, value=mnesia_tid_locks] -- list all mnesia_tid_locks atoms
  • mods.funs.exprs.sub[type=string, value~"Error message.*"] -- list all strings which contains the "Error message" string
  • mods[name=mnesia_log].funs.exprs.sub[type=tuple,[.sub[index=1].origin[type=atom,value =backup_args]]] -- list all functions which was called with a tuple as an argument containing backup_args atom as a first argument (foo({backup_args, Sth1, Sth2}, Sth3))
  • mods[name=mnesia_log].funs[name=open_log].refs
  • @expr.origin
  • ri:anal_dyn()
  • mods.funs.dynrefs
  • mods[name=mnesia].funs[name=write, arity=1].dynrefs
  • @fun.dynrefs
  • mods[name=mnesia_log].funs[name=open_log].refs[.param[index=1].origin[type=atom, value=decision_log]]
  • mods[name=mnesia].funs[name=foldr].called_by
  • mods[name=mnesia].funs[name=foldr].called_by[mod /= mnesia] -- list the calls to the foldr function from an other module than the defining mnesia
  • mods.macros -- list alldefined macros
  • mods.macro[name="DEBUG_TAB"].refs -- list the usages of the DEBUG_TAB macro
  • @macro.refs -- list the appications of the pointed macro
  • @expr.macro_value -- show the expanded value of a macsro application

Checking design rules

  • mods.funs.is_tail_rec -- list whether a function is tail recursive
  • mods.funs[loc>50] -- lists long functions
  • mods.funs[max_depth_of_cases>3] -- list the functions that are too deeply nested
  • mods.funs[branches_of_recursion>5] -- list those functions that considered to complax and has more than 5 recursive branches
  • mods[max_length_of_line>80] -- lists the modules containing lines longer than 80 characters

Detecting vulnerabilities

  • mods.unsecure_calls -- Lists all the possible vulnerabilities
  • mods.unsecure_interoperability -- Lists interoperability related weaknesses
  • mods.unsecure_concurrency -- Identifies concurrency related issues
  • mods.unsecure_os_call -- Checks for OS injection
  • mods.unsecure_port_creation -- Identifies port creation related issues
  • mods.unsecure_file_operation -- Lists unsecure file handling
  • mods.unstable_call -- Shows possible atom exhaustion
  • mods.nif_calls -- Identifies unsecure NIF calls
  • mods.unsecure_port_drivers -- Lists the unsecure ddll usage
  • mods.decommissioned_crypto -- Lists the legacy functions from crypto module
  • mods.unsecure_compile_operations -- Shows unsecure compile/code loading related operations
  • mods.unsecure_process_linkage -- Lists unsecure process linkage
  • mods.unsecure_prioritization -- Identifies unsecure process prioritization
  • mods.unsecure_ets_traversal -- Lists unsecure ETS traversal
  • mods.unsafe_network -- Checks for unsecure kernel related operation
  • mods.unsecure_xml_usage -- Identifies unsecure xml parsing
  • mods.unsecure_communication -- Lists unsecure communication related settings