How to use NLP (Natural Language Processing) libraries for post-processing Event Stormings in Miro

Within the framework of our Innovation Incubator, the idea arose to develop some tools to facilitate the post-processing of Event Storming sessions conducted in Miro. These tools should help extract information from event stormings for further analysis. In a previous blog post, we discussed how to extract data from Miro Boards. In this blog post, I will show you which text preparation steps are necessary to be able to analyse sticky notes from event stormings and how this analysis can be carried out with the help of the spaCy library.

Motivation

Possible questions that are worth exploring during the post-processing of Event Storming sessions include:

• How many domain events/commands etc. were created?

• How are they distributed across the timeline?

• Are there domain events that occur several times?

• What are the main subjects?

• Who was the most active participant in a specific bounded context?

To address questions like these, it is essential that specific DDD concepts are represented correctly by the sticky notes. Event Storming uses a very specific colour-coded key. We represent domain events with orange sticky notes, while commands are written on blue ones. On the other hand, the verb form is crucial. Domain events are described by a verb in past tense (e.g., “placed order”) and commands, as they represent intention, in imperative (e.g., “send notification”). These conventions sound simple, but once participants are “in the flow”, they sometimes have a hard time sticking to them. Restricting ourselves to domain events for now, that means, that we have to ensure in post-processing

a) that all orange sticky notes contain a past-tense verb and

b) that stickies containing past-tense verbs are orange

Luckily, there are NLP libraries that make the analysis of verb morphology (= the study of the internal structure of verbs, e.g. past tense suffixes such as -ed) easy. In the next section, I will explain how this was done using spaCy.

Example of partially incorrectly formulated sticky notes

Analysis of verb morphology using spaCy

The starting point was a pandas data frame with information about colour, creator and the content of the sticky note (see here for an explanation of how to retrieve data from Miro).

However, before most NLP tasks, it’s necessary to clean up the text data using text preprocessing techniques. For this purpose, we did case normalization and removed punctuation. (here, Python's re module, which provides regular expression matching operations comes in handy).

for i in range(len(df)):
    df.at[i, 'text'] = re.sub(r"[^\w]", " ", df.at[i, 'text']).lower()

The next necessary step is tokenisation. Tokenisation is the process of breaking down a piece of text into smaller, meaningful units like sentences or, as in our case, words. Therefore, we first imported the spaCy library and then loaded its English language model. Tokenisation can then be done by iterating over the doc objects.

nlp = spacy.load("en_core_web_sm")

def analyse_tense(df):
    for j in range(len(df)):
        doc = nlp(df.at[j, 'text'])
        my_dict = {}
        for token in doc:
            my_dict[token] = spacy.explain(token.tag_)
            if 'verb, past tense' not in my_dict.values() and 'verb, past participle' not in my_dict.values():
                df.at[j, 'verb_in_past_tense'] = False
            else:
                df.at[j, 'verb_in_past_tense'] = True
    return df

After tokenisation, spaCy can parse and tag a given doc. SpaCy has a trained pipeline and statistical models, which enable it to make a classification of which tag or label a token belongs to, based on its context. The tags are available as Token attributes. Morphological features are stored in Token.morph, but can also be retrieved via the spacy.explain() function.

Evaluation

We first applied the function to orange sticky notes.

orange_stickies = df.loc[df["color"] == "#ff9d48"].reset_index(drop=True)
non_orange_stickies = df.loc[df["color"] != "#ff9d48"].reset_index(drop=True)

analyse_tense(orange_stickies) 
analyse_tense(non_orange_stickies)

It turned out that it worked well to identify domain events that do not contain verbs in past tense (only in one case, where the verb was misspellt, it failed to recognize the token as a verb).

When analysing all other sticky notes that contain past-tensed verbs, it becomes apparent that the function correctly indicates that the “bought popcorn” note should actually be orange. However, it also incorrectly identifies past participle forms used as an adjective as a verb.

Conclusion

Overall, you can see that NLP libraries like spaCy can help you with the post-processing of Miro event stormings, as you can use them to single out those sticky notes that need to be checked. To further improve the result, one could consider to include spelling correction in the step of text preparation (e.g., using the TextBlob library).