Why do ecommerce agencies often code with tons of errors?
In this article, I’ll answer the question: why do ecommerce agency implementations often have bugs and errors, and why does the client feel like a tester working for free for the agency?
Why does what an ecommerce agency implements for you have a lot of errors?
An ecommerce store from the consumer electronics & appliances sector
Symptoms that something is wrong with the quality of an ecommerce agency’s work
What are the consequences of working with a software house that generates errors in code work?
Why does the work implemented by a software house have a mass of errors?
What a correct testing process (Quality Assurance) should look like?
What are critical processes?
What should happen after a production launch?
What should you look for when talking to a software house or ecommerce agency?
What to do in a situation where what an ecommerce agency implements for you has a ton of errors?
Summary
The ecommerce agency you are working with probably does not have developed testing processes. It probably doesn’t have testers, and the project coordinator carries out the tasks of checking the functionality coded by developers.
What then should you do in such a situation? Consider changing the ecommerce agency with one that has developed quality assurance processes. This is a last resort, but I can’t imagine that you, as a client, should wait to improve the quality of your ecommrece. If you already launched your new online shop, you should conduct an audit and perform manual tests of all critical processes. Then, the agency should write automated tests for these processes so that ecommerce agency can react quickly when a major error occurs.
In a recent meeting with an ecommerce director (from the extremely competitive consumer electronics and appliances industry), he admitted that he had serious problems with his current ecommerce agency.
Regardless of the agency’s performance, every task they complete is with errors.
I decided to explore this topic further. I asked some additional questions to understand this ecommerce director’s problem better.
In the beginning, I asked what the hours’ breakdown in the ecommerce agency’s monthly reports provides him. How many hours are described as bug fixing, and how many hours are allocated to development? It turned out that ecommerce allocates 160 hours per month to development work the ecommerce agency performs. Of this pool, as many as 90 hours are spent on bug fixing. The remaining 70 hours are used for development topics. So here we have a somewhat strange split: 90 hours for bug fixing and 70 hours for development.
During the interview, I also asked about user stories and acceptance criteria. In the beginning, Bob (e-commerce director) wasn’t sure what these were, so I explained that user stories and acceptance criteria describe a given ecommerce functionality. They are usually prepared during the pre-implementation analysis phase (also called the discovery phase), which precedes the coding done by developers.
The idea is that the functionality should not be described in vague terms and that all parties should know exactly how it should work. All these assumptions should be described as acceptance criteria. The process is much simpler when everyone involved in the project has a clear idea of the functionality. Additionally, with clearly defined acceptance criteria, testing the site is possible. This is because there are no clear guidelines according to which functionality should be tested. An example that comes to mind is when someone says that a car “goes fast.” For one, it will be 50 km/h, and someone else imagines a car speeding 140 km/h. The point is that when describing how a part of an ecommerce system should work, one should not use such subjective criteria. The car is supposed to move fast, at a speed of no less than 120 km/h-now it is clear to everyone.
Let’s go back to Bob. I asked him how much time per week he spends testing and describing bugs for his ecommerce agency. I was surprised to learn that it’s between 8 and 12 hours! That’s one to one and a half days a week for testing software house tasks, describing bugs, and communicating with that partner’s team—quite a lot.
Let’s start by understanding what symptoms in Bob’s situation indicated that the ecommerce agency’s work quality could be better.
A software house would deliver functionality, and after a quick test, it would turn out that the functionality didn’t work properly. Briefly and to the point.
The second symptom was that one thing was corrected, and another suddenly stopped working. For example, after implementing the ability to place an order as a guest (without having to register), it suddenly appeared that one could not log in to an already-created accounts. Although the two functions were unrelated, the introduction of one somehow affected the operation of the other.
There were not many errors during the implementation of the online store. The problem started after the site went live. When the site was made available to end customers, a lot of errors appeared.
Another important aspect is the budget spent on fixing bugs. In Bob’s case, well over 50% of the working hours are taken up by fixing bugs and patching holes.
If cooperation with an ecommerce agency looks like this, you can expect several unforeseen consequences.
First of all, a lot of time is lost. Eight to 12 hours a week for checking, documenting errors, describing, and communicating is a lot. You don’t pay the ecommerce agency for these hours, but they are a cost that your team (or you) bear.
The second issue is lost revenue. If a key ecommerce functionality (for example, you can’t select a delivery method) fails randomly, it may prevent some customers from placing orders.
Errors also increase the time to market new ecommerce functionality. For example, the functionality of the new presentation of discounts (which we wanted before Black Friday) is handed over to the ecommerce agency for testing on 28/10. After 5 minutes of testing, you already know it does not work properly. Now, you have to describe the errors in detail and send a message to the agency. This one has to analyze the error, fix it, and submit it again for your testing. You will probably find that you will not make it with this functionality in time.
The software house you work with delivers functionality that contains a lot of errors for several main reasons.
The first reason is the lack of a tester on the project or project team working for you on the ecommerce agency side (alternatively, there is one tester for a team of 20 developers). In many agencies, a tester is often considered an unnecessary budget line. During the sales process, this cost line makes an agency’s offer a dozen/tens of percent more expensive. Instead of a tester, the project coordinator (project manager) tries to check what the developers hand over to him for testing.
He has to do this while managing the project, managing the project team, and communicating with you and other clients. This is an impossible task to accomplish. The project coordinator also often doesn’t have the competencies and processes for testing and doesn’t know the tools that speed up the testing process.
The second reason your projects may contain many bugs is that the budget allocated for testing is insufficient. For example, if you set aside 160 hours of developer work each month, you should allocate at least 20% of that time to testing.
20% is the absolute minimum. During this 20%, the tester is able to test the functionality on one browser and two break points: mobile and desktop.
If the ecommerce agency shows that it allocates about 5% of a developer’s time per month to testing, no tester is working on your project.
Additionally, if you want to cover more technologies (browsers) and breakpoints with testing, you should allocate more time. If it seems like a lot to you, it’s worth comparing it to the cost: fixing bugs + the cost of your time + lost revenue.
The last point concerns the “source” of errors. Many are due to the lack of user stories (storekeepers) and acceptance criteria describing functionality. If you haven’t received accurate user stories and acceptance criteria after the pre-implementation and analysis phase, your project probably won’t have a tester. This is because the tester does not have criteria by which he should test the online store or the functionality in question.
If you have doubts whether the documentation you received after the pre-implementation stage is sufficient, just write to us. We will suggest whether what you have is sufficient so that you can carry out programming work on this basis.
Then, let’s consider how it should look to make everything work correctly.
Let me introduce 3 key concepts: code review, manual, and automated testing.
Code review is the checking of code written by one programmer by a more experienced programmer. It is worth noting that code review does not check whether the functionality visible on an online store works as originally intended. It is only a check of the structure of the code and how it was written.
Then we have manual testing. This is when the tester manually checks whether the website meets the acceptance criteria. For example, if it is written that a certain page should appear when clicked, that there should be certain fields to fill in during registration, and that validation is supposed to work in a certain way, the tester checks that it all works that way. If yes, the acceptance criterion is met. If not – it is not met. The task must be corrected (realized once again by the programmer).
Automated tests is, in simple terms, a program/script that attempts to execute a certain process on an online store by checking its correctness. It can be programmed, for example, to add a product to the shopping cart, select a form of payment, log into an account, and make a purchase. The whole process is automatic; you just need to press a button to test the whole process. The biggest advantage is that you can write automatic tests for the most critical processes and validate them, for example, once a day.
In developing an ecommerce project, it is advisable to involve a tester as early as the pre-implementation analysis (discovery phase). This may seem unnecessary since the online store does not yet exist, but… removing errors that result from incorrect assumptions is several times cheaper at this stage than if it were already done after coding.
Once we have the acceptance criteria defined and the programmer has coded the task – for example, the search engine is already running on your store – the tester can manually check the functionality.
He checks that the functionality works as described and meets the acceptance criteria – this is the most important thing. The tester checks it all point by point on all resolutions (at least mobile and desktop). If something doesn’t work properly, the functionality returns to the programmer for correction.
The functionality is passed on to you only after receiving a positive evaluation from the tester. You get a proven, tested functionality in which you are unlikely to find errors.
About 2 months before the website goes into production, automated tests should be written for critical processes, that is, the most important processes in ecommerce.
Examples of 5 such processes are login, registration, adding a product to the shopping cart, recalculating prices in the shopping cart and the value of the entire order, and placing the order.
These are critical processes that must work correctly. If they don’t function flawlessly, then we will seriously hinder users from using our site. It would be worthwhile if there were automated tests written for them to check daily that everything is working correctly.
We are approaching the moment of launching your online store. What happens after the store is available to customers?
We should already have automated tests written and running for critical processes at this stage.
As you develop your store and new functionalities appear, a tester should test each before it goes to you. There should be user stories and acceptance criteria before testing so that the tester has clear guidelines on how to test the functionality and be sure that it meets your requirements.
It is also important for the ecommerce agency to create a cockpit with the most important life parameters of the website. This should include basic metrics from Google Analytics, such as rejection rate, conversion rate, site speed, and load time. This allows the ecommerce agency to set up warnings, or so-called alerts, if the level of these metrics drops dangerously. This signals that something troubling may be happening with the site.
Here’s how it should look. Ask about their testing processes if you are interviewing an ecommerce agency or software house.
They will describe a process similar to the one I outlined in this article. However, something is wrong if someone says they have testers, but there is no need for documentation related to acceptance criteria.
If the process deviates significantly from the one I described, or the person you are talking to cannot describe the process, then you should approach the collaboration with great caution.
Finally, I’d like to share with you the action plan I developed for Bob (who is Bob? An ecommerce director whose story I described at the very beginning)
I recommended conducting an audit before starting any cooperation. The audit will allow you to check the backend and frontend performance, the code quality, and how the finished modules are implemented.
In terms of testing, the most important thing is to manually test key (critical) processes.
Then, our plan was to write automated tests for these critical processes. It will take about 40-60 hours to write 5 tests for critical processes.
The final step is to prepare a cockpit that will monitor key metrics and site parameters. The cockpit will also send automatic alerts when something starts malfunctioning. This will allow you to react quickly and solve the problem before site users notice it.
To sum up Bob’s situation, is it possible to fix all the errors on the site that arose during the implementation stage? The answer, unfortunately, is NO.
Errors made during implementation can be numerous and costly to fix, so you will have to live with some of them. The proposed plan of action will lead to the most important and serious errors being removed, and, in addition, the system of tests and cockpits will allow for a quick response if errors have already appeared. Errors could be avoided if, during implementation, the ecommerce agency had a solid quality assurance processes.
Why does what an ecommerce agency implements for you have a ton of errors? The most common reason for such situations is the lack of testing processes to ensure the quality of the functionality created by the ecommerce agency.
What action should you take in such a situation? Although it may seem like a drastic solution, most often, the best option is to change the ecommerce agency. This is because finding a constructive and quick solution to the problem is difficult. This, from the time that you tell the agency, “Hey, you don’t have testing processes and a tester. Change it.” to actual change will take at least 6 months. The question is whether you want to wait that long.
Sounds like cheap advertising, but if you are facing a large number of bugs in your project, in functionalities that an ecommerce agency codes, then why not have a conversation? During the conversation, we would investigate what exactly the problems are, and as a result, we would work out concepts to solve your situation.
