Changes between Version 3 and Version 4 of Dependency


Ignore:
Timestamp:
Mar 20, 2012, 8:29:48 AM (13 years ago)
Author:
manualwiki
Comment:

dep. new front page

Legend:

Unmodified
Added
Removed
Modified

  • Dependency

    v3 v4  
    1 = Module and Function Dependencies = 
     1= Dependency analysis = 
    22 
    3 == Concerning terminology == 
    4 We say that a module A is '''dependent''' on another '''module''' ''B'' (''A -> B'') if there is 
    5 at least one function call from ''A'' to ''B''. \\ 
    6 A '''cyclic dependency''' appears, when ''B'' is 
    7 also dependent directly (''B -> A'') or indirectly (e.g. ''B -> C -> A'') from ''A''. \\ 
    8 Note that it is possible to have a cyclic dependency among the modules while having 
    9 no cyclic dependencies among the functions. \\ For example, a function call from 
    10 ''A:foo'' to ''B:foo'', and from ''B:foo2'' to ''A:foo2'' implies a cyclic dependency on the 
    11 module level. \\ 
    12 If one wants to have a deeper analysis and pays more 
    13 attention to the concerning functions, a '''function''' level query should be done. \\ In 
    14 our previous example, no function level cycle appears, unless ''A:foo'' calls ''B:foo'', 
    15 and ''B:foo'' calls ''A:foo''. \\ 
    16 \\ 
     3The base of dependency analysis is function calls. \\ 
    174 
    18 == Possible Analysis == 
     5Basically, the following examinations can be done: 
     6 * cycle checking and listing 
     7 * relationship search 
     8 * drawing 
    199 
     10On every level these operations have different subtypes which will be described in details at the appropriate section. 
    2011 
    21 The following examinations can be done considering dependencies:\\ 
    22 * ''Checking'' whether there are ''cycles'', if so, ''listing'' them out 
    23 * ''Printing out'' the cycles, meaning the modules/functions will not be represented by their proper graph node, but with their ''names'' (instead {{{ {'$gn', module, 3} }}} there will be {{{test}}}). 
    24 * Checking for cycle ''from one or more nodes'' as starting points 
    25 * ''Drawing'' the dependency graph 
    26 * Drawing the dependency graph from a starting node 
    27 * Drawing the cyclic part of the dependency graph if one exists (you can also give a cyclic node as a starting node) \\ 
    28 \\ 
    29 Dependency analysis can be done through '''two''' interfaces: \\ 
    30  
    31 1. ''ri'' (using RefactorErl through console interface) 
    32 2. using the ''web'' interface. \\ 
    33  
    34  
    35 The two interface functions are: 
    36  
    37 1. {{{ri:draw_dep/1}}} - for drawing 
    38 2. {{{ri:print_dep/1}}} - for listing, printing out to the standard output \\ 
    39 \\ 
    40 \\ 
    41 \\ 
    42 To call the desired query, the user should give a ''proplist'', stating the different requirements. \\ 
    43    The options and the keys for the functions are: 
    44 * {{{{level, Level} }}}  
    45     {{{Level = mod | func}}} 
    46   Stating the level of the query, module or function level. 
    47 * {{{{type, Type}}}} \\  
    48      {{{Type = all | cycles}}} 
    49    The investigation should be done on the whole graph/table, or just on the cycle part (if it exists). \\ 
    50    When listing out the cycles, {{{all}}} gives back the result in their graph node form, while {{{cycles}}} returns with their proper 
    51    names. 
    52  
    53  
    54 === Examples for listing results === 
    55  
    56  
    57  
    58 * Checking for cycles in module level. \\ 
    59   {{{ri:print_dep([{level, mod}, {type, all}]). }}} 
    60    
    61  
    62 * Checking for cycles in function level, and printing out the whole names of the functions (Module:Function/Arity). \\ 
    63  {{{ri:print_dep([{level, func}, {type, cycles}]). }}} 
    64  
    65   {{{{6 cycle(s), }}} \\ 
    66   {{{[['foo:fv4/1','foo:fv4/1'],}}} \\ 
    67   {{{['test3:p/1','test:fv6/1','test3:p/1'],}}} \\ 
    68   {{{['cycle4:f4/1','cycle3:f3/1','cycle4:f4/1'],}}} \\ 
    69   {{{['cycle2:fv2/1','cycle1:fv1/0','cycle2:fv2/1'],}}} \\ 
    70   {{{['test:fv5/1','test:fv4/2','test:fv5/1'],}}} \\ 
    71   {{{['cycle4:f5/1','cycle3:f6/1','cycle4:f5/1']] }}} \\ 
    72  
    73  
    74 * Checking for cycles in function level, and printing out the graph nodes of the functions (notice the changed "type" in the proplist) \\ 
    75  {{{ri:print_dep([{level, func}, {type, all}]).}}} 
    76  
    77   {{{{"6 cycle(s)",}}} \\ 
    78   {{{[[{'$gn',func,28},{'$gn',func,28}], }}} \\ 
    79   {{{[{'$gn',func,29},{'$gn',func,37},{'$gn',func,29}],}}} \\ 
    80   {{{[{'$gn',func,7},{'$gn',func,9},{'$gn',func,7}],}}} \\ 
    81   {{{[{'$gn',func,2},{'$gn',func,1},{'$gn',func,2}],}}} \\ 
    82   {{{[{'$gn',func,36},{'$gn',func,35},{'$gn',func,36}],}}} \\ 
    83   {{{[{'$gn',func,8},{'$gn',func,6},{'$gn',func,8}]]}}}} \\ 
    84  
    85 * Checking for cycles in module level from an exact node \\ 
    86  {{{ri:print_dep([{level, mod}, {gnode, {'$gn', module, 24}}]). }}} 
    87  
    88          {{{{true,[[{'$gn',module,24},}}} \\ 
    89          {{{{'$gn',module,25},}}} \\ 
    90          {{{{'$gn',module,24}]]}}}} \\ 
    91  
    92  
    93 * Checking for cycles in function level from an exact node given with its whole name \\ 
    94  {{{ri:print_dep([{level, func}, {gnode, "cycle4:f5/1"}]). }}} 
    95  
    96 == Representation == 
    97  
    98 === Function Level === 
    99 Let's take an example to explain the meaning of the representation of 
    100 dependency graphs.  A {{{ri:draw_dep([{level, func}, {type, all}])}}} 
    101 call was made, which generated a ''Graphviz dot file''. \\ 
     12The analysis can be done on different levels: 
     13 * [wiki: Function] 
     14 * [wiki: Module] 
     15 * [wiki: Function blocks] 
    10216  
    103 !!TODO: add figure!! 
    104  
    105 Explanation of the figure: 
    106  
    107 * ROOT triangle - no actual purpose, functioning as a starting point, only appears in the representation 
    108 * Rectangle/box nodes (eg.: {{{cycle1, a, test2}}}) - representing modules (colour: ''deep purple'') 
    109 * Hexagon nodes (eg.: {{{f1/1, apply/3, test2/2}}}) - representing functions (colour: ''black'') 
    110 * Double octagon nodes - representing opaque nodes (colour: ''black'', label colour: ''gray'') 
    111 * Solid, continuous edge, normal arrowhead - normal edge indicating the modules from the root, and the 
    112         modules and their definitions of functions (colour: ''black'') 
    113 * Dashed edge, normal arrowhead - indicates that a function calls another function ({{{funcall}}}) (colour: ''black'') 
    114 * Dashed edge, special arrowhead - indicated a function call, but also that it is a cyclic edge (colour: ''red'') 
    115  
    116  
    117 The next figure was made after a {{{ri:anal_dyn()}}} was run, which is a dynamic analyser. Due to its work the call graph changes a bit,  
    118 new types of nodes and edges are introduced. 
    119  
    120 !!TODO: add figure!! 
    121  
    122 Explanation of figure, new nodes, edges: 
    123  
    124 * Double octagon nodes - representing opaque, -1 arity nodes (colour: ''black'', label colour: ''gray'') 
    125 * Dotted edge, normal arrowhead - indicates an {{{ambcall, dyncall, may_be edge}}} (colour: ''black'') 
    126  
    127 Every node and edge have tooltips. In the case of nodes it shows the 
    128 adequate graph node, while considering edges it depicts the type of 
    129 function call ({{{funcall, may_be, ambcall, dyncall}}}). Static calls are labelled by {{{funcall}}}. 
    130  
    131 {{{Dyncall}}} means unambiguous dynamic calls, when the identifiers 
    132 of the callee may be defined not at the call but at another program 
    133 part, and their values are can be calculated by data-flow 
    134 reaching. {{{Ambcall}}} relation denotes the fact that some of the 
    135 arguments of the dynamic function call can not be detected 
    136 statically. If the number of parameters is uncertain, then after the 
    137 arity of the function will be -1. It is also indicated with a special 
    138 node. For further details about the dynamic calls and their 
    139 representation can be found in (TODO: Reference for dynamic calls). When the 
    140 applied function or module is uncertain, the tool represents it with 
    141 an {{{opaque}}} node, and connects the possible functions/modules 
    142 with {{{may_be}}} edges.  The representation in the analysis now 
    143 follows this, indicating the opaque nodes differently (described in 
    144 the previous section, concerning drawings). 
    145  
    146 We note here that tooltips may not be shown under certain browsers 
    147 (for example Mozilla Firefox 7.0.1). If this happens, please try 
    148 another browser. 
    149  
    150  
    151 === Module Level === 
    152  
    153 Similarly, at module level, after calling {{{ri:draw_dep([{level, mod}, {type, all}])}}, an analogous figure can be achieved. \\ 
    154  
    155 !! TODO Figure !! 
    156  
    157 Explanation of figure: 
    158 * Rectangle/box nodes (eg.: {{{cycle1, erlang, test2}}}) - representing modules (colour: ''deep purple'') 
    159 * Doubleoctagon nodes (no example on the figure) - representing opaque nodes (colour: ''black'', label colour: ''gray'') 
    160 * Dotted edge, normal arrowhead - indicates that a module calls another module (colour: ''black'') 
    161 * Dotted edge, special arrowhead - indicated a module call, but also that it is a cyclic edge (colour: ''red'') 
    162  
    163  
    164 The tooltip for edges shows a list the function pair: \\ 
    165 {{{Call = {callee, called}, Tooltip = [] | [Call | Tooltip]}}} \\ 
    166 Latter makes a connection between the modules with this call. 
    167  
    168 === Examples for the representation of the results ===  
    169  
    170  
    171 * {{{ri:draw_dep([{level, mod}, {gnode, erlang}]).}}} 
    172 * {{{ri:draw_dep([{level, mod}, {gnode, erlang}, {dot, "/home/working/dot/test.dot" }]).}}} 
    173 * {{{ri:draw_dep([{level, func}, {gnode, "lists:hd/1"}]). }}} 
    174 * {{{ri:draw_dep([{level, mod},{gnode, {'$gn', module, 4}}]). }}} 
    175 * {{{ri:draw_dep([{type, cycles}, {level, func}, {gnode, {'$gn', func, 36}}]). }}} 
    176  
    177 == Function block dependencies == 
    178  
    179 === About function blocks === 
    180 In large systems, sets of applications (which themselves consist of several modules) are organised into bigger units; keeping in line with 
    181 Ericsson terminology, we shall call these '''function blocks'''.  We also seek dependencies between them, which is conceptually similar to 
    182 dependencies between modules: a function block ''FB1'' is dependent on a function block ''FB2'' if a module from ''FB1'' is dependent on one from ''FB2''. \\ 
    183  
    184 In the following, we shall presume that all function blocks reside in directory trees, and that all modules contained in the tree are loaded 
    185 into the database. \\ 
    186  
    187 If we want to sum up what function blocks are, it should be said that they are groups of modules, in which every function block on its own 
    188 attains a certain functionality. However, there can be several concepts how exact function blocks can be distinguished. Three ways will be presented. \\ 
    189  
    190 1. Considering ''every directory'' as a function block is not a far to seek option. The exact ''absolute paths'' of every module manages them into unambiguous units, and between these units module relationships give the basis of the dependency analysis. \\ 
    191 2. Continuing the absolute path logic, one may come to the point when these absolute paths need to be generalised, or following some kind of structure or concept. The tool provides interface for this request as well, since it is possible to define function blocks by the means of ''regular expressions''. In this way, for example every module in the {{{.*/src/}}} directories can be considered as one block, and connections outside of this function block (for instance, relationship investigation with {{{.*/ebin/}}} directories) can be determined. The default definition of function blocks comes from the ''NetSim'' team which is defined as a regular expression. The default rule can be overridden by redefining the regular expression (exact regexp is given in the TODO reference to ''Defining function blocks with regular expressions'' paragraph). \\ 
    192 3. To define a function block is to ''list its contents''. The user can define his own function blocks in 3 ways, with defining the exact modules (eg.: {{{module1, module2, module3}}}, placed in three different directories, can a function block), giving the absolute directory path with/without regular expressions (to give it further thought, the exact absolute path is also a regular expression) and giving structure of the names and absolute paths of Erlang source files with regular expressions (example in TODO refernce {{{User defined function blocks}}}} paragraph). 
    193  
    194  
    195 == Using function block analysis in {{{ri}}} == 
    196 The function {{{ri:fb_relations/1}}} is used for the function block dependency analysis. The argument, which is again a proplist of this function determines the exact examination type. \\ 
    197 The Options are the following: 
    198 * {{{{command, Command}}}} \\ 
    199   {{{Command = get_rel | is_rel | check_cycle | draw | draw_cycle}}} 
    200   * {{{get_rel}}}  
    201     Displays the relationship between the given function block list. The result is a tuplelist where a tuple represents a relation. 
    202   * {{{is_rel}}} 
    203     Decides whether there is a connection between the two given function blocks. 
    204   * {{{check_cycle}}} 
    205     Checks for cycles in the dependencies between the given function block list. Unless list is given, checks among every function block list. 
    206   * {{{draw}}}  
    207     Prints out the entire graph or creates a subgraph drawing from the given function block list. Output file is {{{fb_relations.dot}}}. 
    208   * {{{draw_cycle}}}  
    209     Prints out a subgraph which contains the cycle(s). Unless cycles exist, prints out the entire graph. Output file is {{{fb_rel_cycles.dot}}}. 
    210 * {{{{fb_list, List}}}}, {{{List = [string()] |}}} {{{[{Basename::string(), [Function block::atom()]}]}}} 
    211     Chosen function block lists for further examinations. If no list 
    212     given, then it takes every function block list, which means that 
    213     every absolute path defines a function block. 
    214  
    215 * {{{{other, Other}}}} {{{ Other = bool()}}} 
    216      The {{{Other}}} parameter stands whether the category "Other" would be taken into consideration or not ({{{true/false}}}). The default value is true. 
    217  
    218 === Examples === 
    219  
    220 * {{{ri:fb_relations([{command, check_cycle}]).}}} 
    221 * {{{ri:fb_relations([{command, draw_cycle}]).}}} 
    222 * {{{ri:fb_relations([{command, is_rel}, {fb_list,["path_to_dir/subdir", "path_to_dir/subdir/subsubdir"]}]).}}} 
    223 * {{{ri:fb_relations([{command, is_rel}, {fb_list,{"path_to_dir", [1, 2]}}]). }}} 
    224  
    225 === Optional ''Other'' category === 
    226 Let's think about function blocks in the sense that every directory with its absolute path gives a different function block.  The Other category is a special collector name for those modules which cannot be divided into any function block. Practically this covers those modules which do not have directories (for example, usually erlang). This category can be taken into consideration as a false function block, so 
    227 a new option was introduced for eliminating this category. With this the dependencies with ''Other'' category are skipped, the tool takes into consideration only the real connections. 
    228  
    229 === Representation of Function block Dependency Graphs === 
    230  
    231 Naturally, it is possible to represent function block relationships with the previously used dot files. Using the {{{command}}} key in the ''Options'' proplist, the desired figure can be gained ({{{draw}}} or {{{draw_cycle}}}). It could be useful to make the standard output messages also available, because function blocks are represented with numbers. However, as a tooltip the proper function block name is provided. 
    232  
    233 {{{ >> ri:fb_relations([{command, draw}]). }}} \\ 
    234 {{{Earlier results deleted (except .dot files). }}} \\ 
    235 {{{Building dependency table... }}} \\ 
    236 {{{Creating "/home/RefactorErl/tools/new/tool/dep_files/fb_relations.dot" file... }}} \\ 
    237  
    238 ---------------------------------------------------------- \\ 
    239 {{{Function block 1 is "/home/RefactorErl/test/cyclic/cycles" }}} \\ 
    240 {{{Function block 2 is "/home/RefactorErl/test/regexp/common/serv1/ebin"}}} \\ 
    241 {{{Function block 3 is "Other"}}} \\ 
    242 {{{Function block 4 is "/home/RefactorErl/test/error"}}} \\ 
    243 {{{Function block 5 is "/home/RefactorErl/test/cyclic/no_cycle"}}} \\ 
    244 {{{Function block 6 is "/home/RefactorErl/test/opaque"}}} \\ 
    245 {{{Function block 7 is "/home/RefactorErl/test/regexp/common/serv2/lib/ebin"}}} \\ 
    246 ------------------------------------------------------------\\ 
    247  
    248  
    249 !!! TODO add figure !!! 
    250  
    251 Explanation of figure: 
    252 * Hexagon nodes (eg.: {{{cycle1, erlang, test2}}}) - representing  function blocks as number (colour: ''black'') 
    253 * Dotted edge, normal arrowhead - indicates that a fb calls another fb (colour: ''black'') 
    254 * Dotted edge, special arrowhead - cyclic edge (colour: ''red'') 
    255  
    256 === Using function block analysis from the web interface === 
    257 The usage of function block analysis is described in !!! TODO add ref.!!! 
    258  
    259 === Defining function blocks with regular expressions === 
    260 Function blocks can be filtered by the means of regular expressions. 
    261 An interface function, called {{{ri:fb_regexp/1}} is provided and its parameters are the following: 
    262 * {{{ {type, Type} }}} \\ 
    263   {{{ Type = list | get_rel | cycle | draw }}} 
    264   * {{{list}}} Prints out every function block which matches the 
    265     basic regular expression. 
    266   * {{{get_rel}}} Decides whether there is a connection between 
    267     the two given function blocks. 
    268   * {{{cycle}}} Checks for cycles in the dependencies between the 
    269     given function block list. 
    270   * {{{ draw }}} Prints out the entire graph or creates a subgraph 
    271     drawing from the given function block list.  Output file is 
    272     {{{fb_relations.dot}}} or can be user defined with the ''dot'' 
    273     key. 
    274  
    275 * {{{ {regexp, Value} }}} \\ 
    276   {{{ Value = File::string() | [RegExp::string()] }}} 
    277    
    278   If this option (tuple) is not given, the program works with a basic 
    279   regular expression.  The basic rule: {{{<function block>/common/<service>/ebin}}} or 
    280   {{{<function block>/common/<service>/lib/ebin}}}. \\ 
    281   The regular expression saved for this: 
    282   {{{ (/)[0-9a-zA-Z_./]+/common/[0-9a-zA-Z_.]+/(lib/)?(ebin)$ }}} 
    283    
    284  
    285   * {{{ Value }}} - If the regular expression is given in a file 
    286     then every single regexp has to be defined in a separate line and 
    287     must follow the Perl syntax and semantics as the http://www.erlang.org/doc/man/re.html 
    288     erlang module resembles so. However, the user can give the regular expressions in a list 
    289     as well. If there is an error with a regular expression in the 
    290     file or in the list, it prints out the regexp, the error 
    291     specification, and the position. The most usual regexp is ".*" 
    292     (the Perl syntax does not allow simply "*", because this symbols 
    293     means possible unlimited repetition of characters declared before 
    294     it, and there are no characters declared before it) 
    295  
    296 Examples: 
    297 *  {{{ri:fb_regexp([{type, draw}, {dot, test.dot}]).}}} 
    298 *  {{{ri:fb_regexp([{type, list}, {regexp, "regexp"}]).}}} 
    299 *  {{{ri:fb_regexp([{type, list}, {regexp, "^/home/[a-z./]+}]).}}} 
    300  
    301  
    302 === User defined function blocks === 
    303 One can make his own function block in the following three ways: 
    304 1. Giving the exact modules (with their name) which should be in 
    305   one function block. 
    306 2. Regular expressions covering the structure of the directories. 
    307 3. By regular expressions covering the structure of the directories 
    308   and the structure of the name of the files. 
    309  
    310 Example: 
    311 {{{ refusr_fb_regexp:re([{type, list}, {fb, [[cycle1, cycle2],"/home/user/[a-zA-z]*", "/home/user/[a-zA-z./]*/.*_ui.erl"]}]). }}} 
    312  
    313  
    314 '''Optimisations''' 
    315  
    316 For efficiency and time improving reasons, the result of the queries 
    317 are saved into {{{dets}}} tables for the further queries. 
    318  
    319  
    320 The results of previous queries are saved into dets tables (in the {{{dep_files}}} directory).  This means that if 
    321 the same query is run, the execution time may decreases significantly. 
    322 At first run, a digraph is built as in the previous version, only it 
    323 is saved later.If there was a whole database dependency check, than the tool works from that dets 
    324 table instead of building a new subgraph, which also improves time 
    325 efficiency. Due to this, it is strongly advised that if one knows that 
    326 a lot of different node queries will be done, a whole check should be 
    327 run in the first place.  
    328 The saves are available until the 
    329 database is unchanged. At the moment the hash value of the database is 
    330 changed, the existing dets tables are deleted. The deletion does not 
    331 effect the ''.dot'' files, although it is significant to remember 
    332 to save them somewhere else from the ''dep_files'' directory, 
    333 because the next call for draw function will overwrite the 
    334 corresponding ''.dot'' file. This could be prevented by using the 
    335 feature, that the user can define his own dot file name and absolute 
    336 path.