Behind the scenes of Nagarro’s biggest testing research project

>> If you want to read this blog in German, click here.

I think everyone has experienced it – these sudden inspirations. You are sitting in a nice beer-garden in Vienna with some of your colleagues, and the discussion moves from the daily business and challenges towards the more inspired, almost philosophical aspects of your work. You start serious, then half-jokingly, transition to some seemingly absurd idea and just have fun with it. The next morning, you fondly remember this discussion, and start your day as usual.

At the beginning of 2019, this happened to me and to some of the smartest, most creative minds I know – with one key difference: We took the idea, challenged it, tweaked it, and ran with it.

This story is not so much about the idea itself. For us, it has become a key topic: How to take testing and automation forward, and how to move beyond just automating the execution of test cases – using AI and Machine Learning, if necessary. You can read all about it here and even watch a cute video on it.

This story is about how we went about working on our idea, in the largest, most intense research program that Nagarro Austria has ever executed. It is about inspiration, pitfalls, perseverance, and a tiny bit about the formalities involved.

“We’re going to be researchers now!”

You wake up with this concept in your head, and some specific ideas on how to implement them, on a technical level. Nagarro is not a research institute – we are a services company. We are not a tool-vendor either. This essentially means that our clients rely on us for our innovation capabilities – we develop a lot of cool new stuff, but with a focus to take our customers’ business forward. This type of strategic research is special. But this is also a special idea – so we decided to go ahead!

What do we expect out of this? What are the benefits for our customers? How does it play into our existing portfolio? What skills do we need? Where do we execute this?

To ask and answer these questions, we took a very pragmatic route: We created a proposal. Not just an internal paper, but we took this opportunity to start working on an application for the Austrian Research Promotion Agency (Österreichische Forschungsförderungsgesellschaft, FFG), which thankfully had to include exactly these points – from the scientific value, over partners & team, to a quite detailed plan. We aimed for a three-year program and were focusing on getting the first year approved. We assembled a team of key players to work on this, including an external consulting agency that specialized in this type of work. We had people with a domain background, technical background, backgrounds in research promotion processes and with a background focused on our target market. For us, this meant that we were not working on this as an “internal project” (which, let’s face it, can become slow and unfocused sometimes – the customer always comes first!), but as a formal process focused on a very specific goal – successfully launching a formally accepted research project!

We also reached out to our contacts at the Vienna University of Applied Technology (FH Technikum Wien), where some Nagarrians have the honor of teaching classes, and asked their feedback and guidance on the more science-focused aspects of this research project, and to other potential partners such as Specialisterne, with whom we have a long-running record of collaboration and cooperation. Their members also have specific skills & experience in the domain we were targeting (training neural networks by labeling huge amounts of data).

After this was done, we handed in the proposal and started our work. This was maybe, a bit of a risky move, as the feedback about acceptance of your application is only provided a couple of months after you hand it in, while the clock for the project itself starts ticking immediately.

“We need to learn … everything!”

We started by learning about the more hands-on aspects of implementation: We took up trainings, shared existing knowledge, reached out to our colleagues all over the world and discussed our specific implementation ideas. Also, our partners at the university helped us formulate the scientific problem, and to create proposals for university students to join us in this project. You could feel the energy – everyone was excited to finally start the actual work! Creative discussions, technical spikes, data collection, we did it all with a high level of energy, and people were eager to contribute – so much so, that we needed to closely monitor and manage the contributions. Only the Austrian team members were supported by the FFG, but this did not really change our global way-of-working: some work items were distributed all over the world, others were mainly worked on in Austria.

We also reached out to our customers and found that some of them were quite interested in the topic. Creating an agreement for sharing data (which is essential in the AI/ML field!) and aligning that with both our legal teams was a challenge, but a manageable one – and the real-world data became a cornerstone for our research.

And then, something very natural happened: Summer holidays!

Remember the pre-corona times? Everyone travelled, had awesome vacations at the beach in some exotic country or just travelled for fun. This also meant that progress became a lot slower in these months, and it was hard to do workshops and work-sessions as a full team. And, as many readers might have experienced themselves, Q4 is usually a very busy time. Progress was steady, but slow. Some people were really engaged, but of course, focused mainly on the work packages that were close to their heart. Progress on the other packages had started slipping.

“Are we slowing down?”

Things dramatically picked up in January, however – we cleaned up our backlog together and pushed ahead with full speed. We handed in the mid-year report to the FFG, planned the second half of the research project cleanly, and went to work with renewed energy.

Then something else happened – COVID-19 changed the world.

This disrupted a lot of things for the project. People were working from home, our face-to-face workshops were no longer possible, and our nice and tidy planning pretty much went out the window. Some people had to work a lot more in their client projects. Others had more time because some customers streamlined their projects due to Corona – so we shifted. We onboarded colleagues who had wanted to join for a while but did not have the time to contribute actively. Now, they saved time because their daily commute was no longer a factor and they had an hour or more to contribute each day. It also meant that we had more budget – we had planned for travel and global conference visits which were no longer possible – all that budget could be moved to hands-on work. So, it actually turned out that Corona had a positive impact on the research project.

These factors sped up our progress considerably – as things started becoming more mature, demos and showcases around our concepts and implementations began to emerge and our energy levels rose. We could see our ideas in action now! This again pushed the team to intensify their activities even more.

“We need to cross the (first) finish line!”

Amid all this dynamic work, we had two more things to think about: The final report, and an application for the second year of research. The final report came quite naturally – we needed to cut it down repeatedly, because we kept exceeding the length-limits proposed by the FFG. We also had our cost reporting well under control, as we took great care to set it up cleanly. No issues there.

As for the second year: we had learned a lot here. We changed considerably from our proposed direction when we had first applied: Every single work package was changed, some of them were even completely scrapped and replaced by new focus points. We sparred with our partners on the application and submitted it in time for a Q3 approval.

A key player left the team, with a new key player coming in two months later, who was even more skilled in the AI/ML area. But we had to introduce him completely into the project and the domain of software quality and testing – as all of us know, hand-overs with a time gap in between are not the most efficient when it comes to knowledge-transfer and continuity, but the “new guy” jumped in brilliantly.

Simultaneously, with the launch of the work on the work packages for the second year, it was time to plan our go-to-market. Our showcases were ready and our content was prepared. So we initiated the work with our designers, marketing, sales, and delivery teams. Flyers, presentation decks, blog and magazine articles, mailings and even an animated video needed to be created. It was time to take this great research and development work, its concept, offering and benefits out into the world!

“We learned a lot!”

• Be focused on your topic, but open to new ideas – so many cool things (and even the project itself) came up because of ridiculous tangents and unfocused discussions.
• Get the right partners on board – without the support from other organizations, this project would not have worked as well as it did.
• When dealing with government-supported projects: Keep your cost-logging meticulous! It helps a lot!
• Every crisis, even if it is earth-shattering and tragic, has its opportunities – COVID-19 galvanized our work and in fact, accelerated it!

Find more about the concepts & technology behind AI4T and connect with us to discuss how Nagarro can help you bring even more intelligence and automation into your software development lifecycle! We have exciting stories to tell and can provide a head-start into the future for you and your team.