Signal-Slot diagram using Doxygen and XSLT

Qt signal-slot analysis using Doxygen and XSLT

Sometimes I need to see connections between signals and slots in my Qt projects. It seems like now there is no software than can do this. Many programs can draw call graphs, but none of them works with Qt signals.

I decided to write own C++ parser, but shortly I had released that it is a very complicated task. It will take a lot of time without any appropriate result.

So, I need anythig that can parse C++ files for me. And I found that Doxygen can do it. It produces XML files with class structure description and code listing, that are easy to parse.

Lets start with a couple of classes for exploring Doxygen output:

class SenderClass : public QObject
{
      Q_OBJECT
public:
    explicit SenderClass(QObject *parent = 0);

signals:
    void mySignal_01(int, bool);
};

class RecieverClass : public QObject
{
    Q_OBJECT
public:
    RecieverClass(QObject *parent = 0);
    virtual ~RecieverClass() {}
private slots:
    void on_sc_mySignal_01(int,bool) {}
private:
    SenderClass sc;
};

SenderClass does nothing except emitting mySignal_01 signal. The only action of RecieverClass is connection to this signal:

RecieverClass::RecieverClass(QObject *parent) :
    QObject(parent)
{
    connect(&sc, SIGNAL(mySignal_01(int,QList*)),
            this, SLOT(on_sc_mySignal_01(int,QList*)));
}

Now lets process it with the next Doxygen file:

GENERATE_HTML          = NO
GENERATE_LATEX         = NO
GENERATE_XML           = YES

EXTRACT_ALL            = YES
EXTRACT_PRIVATE        = YES

REFERENCED_BY_RELATION = YES
REFERENCES_RELATION    = YES

This will create xml directory with some XML files. Three of them relates to Sender class:

  • classSender.xml
  • sender_8.cpp
  • sender_8.h

The classSender.xml file contains info about Sender class structure

<?xml version='1.0' encoding='UTF-8' standalone='no'?>
<doxygen xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="compound.xsd" version="1.8.6">
  <compounddef id="classSender" kind="class" prot="public">
    <compoundname>Sender</compoundname>
    <basecompoundref prot="public" virt="non-virtual">QObject</basecompoundref>
    <includes refid="sender_8h" local="no">sender.h</includes>
    <sectiondef kind="public-func">
      <memberdef kind="function" id="classSender_1afbe94a9a42b0101deb08204393ec9669" prot="public" static="no" const="no" explicit="yes" inline="no" virt="non-virtual">
        <type></type>
        <definition>Sender::Sender</definition>
        <argsstring>(QObject *parent=0)</argsstring>
        <name>Sender</name>
        <param>
          <type>QObject *</type>
          <declname>parent</declname>
          <defval>0</defval>
        </param>
        <location file="/home/resu/work/reftest/src/sender.h" line="10" column="1" bodyfile="/home/resu/work/reftest/src/sender.cpp" bodystart="3" bodyend="6"/>
      </memberdef>
    </sectiondef>
    <sectiondef kind="signal">
      <memberdef kind="signal" id="classSender_1a2d121184a23b605453b050f420bf0975" prot="public" static="no" const="no" explicit="no" inline="no" virt="non-virtual">
        <type>void</type>
        <definition>void Sender::mySignal_01</definition>
        <argsstring>(int, bool)</argsstring>
        <name>mySignal_01</name>
        <param>
          <type>int</type>
        </param>
        <param>
          <type>bool</type>
        </param>
        <location file="/home/resu/work/reftest/src/sender.h" line="13" column="1"/>
      </memberdef>
    </sectiondef>
  </compounddef>
</doxygen>

It containt <compounddef> element with several <sectiondef> children for different member and access types:
– private/protected/public variables
– private/protected/public functions
– private/protected/public slots
– signals

Each <sectiondef> includes <memberdef> elements with it’s kind (function, variable, signal or slot), type, arguments and location, i.e. file and lines where it’s body is placed

Another usefull file is sender_8cpp.xml. This is a program listing with highlight options for keywords, strings, etc. We will use it for searching QObject::connect() calls for futher parsing.

Before proceed, it will be helpfull to combine all XML files generated by Doxygen into large one. The next command does it:

saxonb-xslt -s index.xml -xsl:combine.xsl

To get connection data, we need to make the following steps:
– find QObject::connect() calls
– split it into arguments
– obtain types of sender and reciever class, signal and slot members
– build connection list
– merge it with origial all.xml file

As a result, each <memberdef> with @kind equals to 'signal' or 'slot' should get new elements showing what signals or slots it is connected to. Then we easly can use it for diagram drawing, unused member search, etc.

Connection string search

Now we need to find code lines with connection commands. It is easy, because these lines always starts with connect(, maybe with spaces.

So, the matches function can be used with the regexp '^\s*connect\s*\('. Remember that all code is contained in codeline elements splitted into highlight for different literal types (keywords, numbers, etc.). Our string is a simple text, so it is contained in a single highlight element. So, the final XPath query will be

codeline[count(highlight[matches(.,'^s*connects*(')])>0]

But the code line found with this query may not contain the whole connection command, because it often is broken into multiple lines. Fortunately, connect call always ends with semicolon, so its enough to collect all codelines until the semicolon is found.

It is done by code below:

  <xsl:template match="codeline[count(highlight[matches(.,'^s*connects*(')])>0]">
    <xsl:apply-templates select="." mode="findsemicolon"/>
  </xsl:template>

  <xsl:template match="codeline[count(highlight[contains(.,';')])=0]"
                mode="findsemicolon">
    <xsl:param name="value"/>
    <xsl:apply-templates mode="findsemicolon"
                         select="../codeline[@lineno = current()/@lineno+1]">
      <xsl:with-param name="value" select="concat($value, .)"/>
    </xsl:apply-templates>
  </xsl:template>

  <xsl:template match="codeline[count(highlight[contains(.,';')])>0]"
                mode="findsemicolon">
    <xsl:param name="value"/>
    <!-- concat($value, .) is the whole connection command -->
  </xsl:template>

Here we use findsemicolon mode, because otherwice the second and the third templates will match all strings with semicolons.

Connection string parsing

Having the whole connect call, the next step is to extract four parts of this call:
– sender
– signal
– reciever
– signal or slot

The last part can be signal because Qt allows not only signal-to-slot connection, but signal-to-signal too.

XSLT 2.0 introduces very helpful <xsl:analyze-string> element. It allows us to parse the string, capture it’s parts and then create result using regex-group function.

Here is a function that accepts connection string and returns it’s parts. It also says is the connection target signal or slot.

  <xsl:function name="mns:tokenizeconn">
    <xsl:param name="x" as="xs:string"/>
    <xsl:analyze-string select="$x"
       regex="connect((.*),s*SIGNAL((.*)),s*(.*),s*(SLOT|SIGNAL)((.*))(,Qt::.*)?);">
      <xsl:matching-substring>
        <tokens>
          <sender><xsl:value-of select="regex-group(1)"/></sender>
          <signal><xsl:value-of select="regex-group(2)"/></signal>
          <reciever><xsl:value-of select="regex-group(3)"/></reciever>
          <target>
            <type><xsl:value-of select="regex-group(4)"/></type>
            <name><xsl:value-of select="regex-group(5)"/></name>
          </target>
        </tokens>
      </xsl:matching-substring>
      <xsl:non-matching-substring>
        <xsl:message>
          Warning: pre-matched string doesn't match regexp:
          <xsl:value-of select="$x"/>
        </xsl:message>
      </xsl:non-matching-substring>
    </xsl:analyze-string>
  </xsl:function>

Sender and reciever parsing

When sender, reciever, signal and target parts have been extracted, we may analyze them separately. Lets begin with sender and reciever. Both can be just class fields, or method calls, or call of field’s method, etc, or just this pointer. For example:

this->field.method(arg1,arg2)

As you can see, the most common case is field/method chain with sections separated by dot '.' or minus-greater pair '-&gt;'. It may be tokenized via the next regular expression:

^([a-zA-Z0-9_]+)((.*?))?(.|->)?(.*)?$

It extracts the first chain section, it’s parenthesis (it there are), section separator and the rest of the string. Using this regexp, we can write function that splits our call chain into sections:

<xsl:function name="mns:tokmbch">
  <xsl:param name="x" as="xs:string"/>
  <xsl:analyze-string select="$x"
                      regex="^([a-zA-Z0-9_]+)((.*?))?(.|->)?(.*)?$">
    <xsl:matching-substring>
      <token>
        <name><xsl:value-of select="regex-group(1)"/></name>
        <type>
          <xsl:if test="string-length(regex-group(2)) gt 0">method</xsl:if>
          <xsl:if test="string-length(regex-group(2)) = 0">field</xsl:if>
        </type>
      </token>
      <xsl:if test="string-length(regex-group(3)) gt 0">
        <xsl:copy-of select="mns:tokmbch(regex-group(4))"/>
      </xsl:if>
    </xsl:matching-substring>
  </xsl:analyze-string>
</xsl:function>

After the first section extraction, it check if there is a separator after it, and, if there is, recursively splits the rest of input. As a result, it returns the sequence of <token> elements containing name of the member called and it’s type — field or function (don’t confuse with type in C++ meaning — we will find it below).

Current class search

To find a class that current line of code belongs to, we first need to find a method containing current code line. Note that code listing and class declaration belongs to different <compounddef> elements of composed XML file. They can be tied via <memberdef>s element <location> It has three attributes:

  • bodyfile – the full path to the file containing method declaration,
  • bodystart – the first line of the method body,
  • bodyend – the last one.

At first, lets take a file the current code line belongs to:

<xsl:variable name="codeline" select="."/>
<xsl:variable name="file" select="$codeline/ancestor::compounddef"/>

Then lets find a method whose definition is in this file, and that starts before the current code line and ends after it:

    <xsl:variable name="method" select="$codeline/ancestor::doxygen//memberdef[
       location/@bodyfile = $file/location/@file and
       number(location/@bodystart) le number($codeline/@lineno) and
       number(location/@bodyend)   ge number($codeline/@lineno)]"/>

Now we can find a class:

    <xsl:variable name="class" select="$method/ancestor::compounddef"/>

Sender and reciever type detection

Having a current class and sender/reciever splitted into individual member calls, we now can find the class of the last section of the chain. To do it, we find the member with a name and kind (funtion or variable) specified by the first section of the call chain, then get it’s type (for a function it will be return type). This type (that is guaranteed to be a class) can be used to process next section of the call chain.

Lets write a function for it:

<xsl:function name="mns:parsmbch">
  <xsl:param name="tokens"/>
  <xsl:param name="class"/>

At first, take the first token in a chain:

  <xsl:variable name="token" select="$tokens[1]"/>

Then, choose a kind of class member we are going to search.
It will be used for kind attribute.

  <xsl:variable name="mkind">
    <xsl:if test="$token/type = 'method'">function</xsl:if>
    <xsl:if test="$token/type = 'field'">variable</xsl:if>
  </xsl:variable>

Next, we may find a class. If the token’s <name> element is 'this', then desired class is the current method class itself. Otherwice, this class may be found by refid attribute of the <ref> element inside <type> of class member with name equal to chain’s section name:

<xsl:variable name="typeid">
  <xsl:if test="$token/name = 'this'">
    <xsl:value-of select="$class/@id"/>
  </xsl:if>
  <xsl:if test="$token/name != 'this'">
    <xsl:variable name="member"
            select="$class//memberdef[@kind=$mkind and name=$token/name]"/>
    <xsl:value-of select="$member/type/ref/@refid"/>
  </xsl:if>
</xsl:variable>

Here we obtain class ID, not the class itself. It is done because <xsl:sequence> inside a variable will create a copy of the node, so we loose a node’s parent.

Now obtain the class definition itself:

<xsl:variable name="type" select="$class/../compounddef[@id=$typeid]"/>

If the token we process is the last in the chain, return it. Otherwise, recursively call function with the found class as a owner for the next token:

  <xsl:if test="count($tokens) = 1">
    <xsl:sequence select="$type"/>
  </xsl:if>
  <xsl:if test="count($tokens) > 1">
    <xsl:sequence select="mns:parsmbch($tokens[fn:position()>=2],$type)"/>
  </xsl:if>
</xsl:function>

Signal and slot type detection

Now lets look at the signal and slot signatures. We already obtained it when parsing the whole connection line. We have <signal> element with a signal signature and <target> element with it’s signature and type (is it a signal or a slot).

<xsl:function name="mns:parssgsl">
<xsl:param name="sgsl"/>
<xsl:param name="class"/>

Introduce a variable target for processing both name/type pair and a single name the same way:

<xsl:variable name="target">
  <xsl:if test="count($sgsl/name) gt 0">
    <xsl:sequence select="$sgsl/*"/>
  </xsl:if>
  <xsl:if test="count($sgsl/name) = 0">
    <name><xsl:value-of select="$sgsl"/></name>
    <type>signal</type>
  </xsl:if>
</xsl:variable>

Split target string in two parts: name and arguments:

<xsl:variable name="nmargs">
<xsl:analyze-string select="$target/name"
   regex="([a-zA-Z0-9_]+)((.*))?">
  <xsl:matching-substring>
      <name><xsl:value-of select="regex-group(1)"/></name>
      <args><xsl:value-of select="regex-group(2)"/></args>
  </xsl:matching-substring>
  <xsl:non-matching-substring>
    <xsl:message>
      Warning: pre-matched string doesn't match regexp:
      <xsl:value-of select="$target/name"/>
    </xsl:message>
  </xsl:non-matching-substring>
</xsl:analyze-string>
</xsl:variable>

Now find a member with these name and type:

<xsl:sequence select="$class//memberdef[
                        @kind=lower-case($target/type) and
                        name=$nmargs/name]"/>

Emit connection information

Now return to the main part of our XSLT. We already obtained all information about connection.

<xsl:template match="codeline[count(highlight[contains(.,';')])>0]"
              mode="findsemicolon">
  <xsl:param name="value"/>

Find code line’s file, method and class, as we did above:

<xsl:variable name="codeline" select="."/>

<xsl:variable name="file" select="$codeline/ancestor::compounddef"/>

<xsl:variable name="method" select="$codeline/ancestor::doxygen//memberdef[
   location/@bodyfile = $file/location/@file and
   number(location/@bodystart) le number($codeline/@lineno) and
   number(location/@bodyend)   ge number($codeline/@lineno)]"/>

<xsl:variable name="class" select="$method/ancestor::compounddef"/>

Split connect() call:

<xsl:variable name="tokens" select="mns:tokenizeconn(concat($value, .))"/>

Obtain sender class:

<xsl:variable name="sendertokens" select="mns:tokmbch($tokens/sender)"/>
<xsl:variable name="senderclass" select="mns:parsmbch($sendertokens, $class)"/>

Get a signal member:

<xsl:variable name="signal" select="mns:parssgsl($tokens/signal, $senderclass)"/>

Do the same for reciever and target

<xsl:variable name="recievertokens" select="mns:tokmbch($tokens/reciever)"/>
<xsl:variable name="recieverclass" select="mns:parsmbch($recievertokens, $class)"/>
<xsl:variable name="target" select="mns:parssgsl($tokens/target, $recieverclass)"/>

Emit connection data

 <connection>
    <sender>
       <class><xsl:value-of select="$senderclass/@id"/></class>
       <signal><xsl:value-of select="$signal/@id"/></signal>
    </sender>
    <reciever>
       <class><xsl:value-of select="$recieverclass/@id"/></class>
       <member><xsl:value-of select="$target/@id"/></member>
       <name><xsl:value-of select="$target/name"/></name>
       <type><xsl:value-of select="$target/@kind"/></type>
    </reciever>
    <string><xsl:value-of select="concat($value,.)"/></string>
 </connection>
</xsl:template>

Now, run XSLT
saxonb-xslt -s:all.xml -xsl:conn.xsl -o:conns.xsl

Merging connection data with original file

After connection list generation, lets merge this data with original composed XML file.

First, read connection data to the variable:

<xsl:variable name="connections" select="doc('connections.xml')/connections"/>

Now, define identity transform for the whole document

<xsl:template match="@*|node()">
  <xsl:copy>
    <xsl:apply-templates select="node()|@*"/>
  </xsl:copy>
</xsl:template>

For slot members, add recievesfrom tags with ID of signal that this slot recieves:

<xsl:template match="memberdef[@kind='slot']">
  <xsl:copy>
    <xsl:apply-templates select="node()|@*"/>
    <xsl:call-template name="addrecieves"/>
  </xsl:copy>
</xsl:template>

Signals can both send itself and recieve another signals, so add recievesfrom and sendsto tags for it:

<xsl:template match="memberdef[@kind='signal']">
  <xsl:copy>
    <xsl:apply-templates select="node()|@*"/>
    <xsl:call-template name="addrecieves"/>
    <xsl:call-template name="addsends"/>
  </xsl:copy>
</xsl:template>

To add signals that current target recieves, find all connection with reciever equals to the current member ID:

<xsl:template name="addrecieves">
  <xsl:apply-templates
    select="$connections/connection[reciever/member=current()/@id]/sender"/>
</xsl:template>

<xsl:template match="sender">
  <recievesfrom>
    <xsl:value-of select="signal"/>
  </recievesfrom>
</xsl:template>

In the same way add targets the current signal sends to:

<xsl:template name="addsends">
  <xsl:apply-templates
    select="$connections/connection[sender/signal=current()/@id]/reciever"/>
</xsl:template>

<xsl:template match="reciever">
  <sendsto>
    <xsl:value-of select="member"/>
  </sendsto>
</xsl:template>
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s