The State of Browser Testing

Automated browser testing is a foundational technology that many web developers rely on. Modern tooling lets a single test drive Chrome, Firefox, and Safari the same way, a triumph of software engineering built on years of standardization work. That standardization emerged from the interplay between open source projects, browser vendors, and standards bodies like the W3C. David Burns https://x.com/AutomatedTester is a longtime Selenium and WebDriver contributor, a veteran of Mozilla, and is currently the Head of Developer Advocacy and Open Source at BrowserStack https://www.browserstack.com/. In this episode, David joins Josh Goldberg to discuss how open source projects, browser vendors, and standards bodies fit together, a practical philosophy of testing built around one solid test per critical path, and the risks that AI and vibe coding introduce to web development. Sponsorship inquiries: [email protected] mailto:[email protected] The post The State of Browser Testing https://softwareengineeringdaily.com/podcasts/the-state-of-browser-testing/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.

• 0:00Speaker 00

  Are you passionate about software development and the tech industry? Software Engineering Daily is looking for a new podcast host to grow its hosting team. In this role, you'll help shape the show's editorial direction and interview engineers, founders, hackers, and tech leaders. Podcasting experience is a plus, but not required. Curiosity, great communication skills, and a genuine interest in the craft of building software are what matter most. If this sounds like you, reach out at [email protected]. Automated browser testing is a foundational technology that many web developers rely on. Modern tooling lets a single test drive Chrome, Firefox, and Safari the same way.

• 0:45Speaker 00

  A triumph of software engineering built on years of standardization work. That standardization emerged from the interplay between open source projects, browser vendors, and standards bodies like the W3C. David Burns is a longtime Selenium and Web Driver contributor, a veteran of Mozilla, and is currently the head of Developer Advocacy and Open Source at BrowserStack. In this episode, David joins Josh Goldberg to discuss how open source projects, browser vendors, and standards bodies fit together. A practical philosophy of testing built around one solid test per critical path, and the risks that AI and Vibe coding introduce to web development.

• 1:28Speaker 00

  This episode is hosted by Josh Goldberg, front-end developer at Sentry and open source maintainer. Josh works on projects in the TypeScript ecosystem, most notably TypeScript ES-Lint, a powerful static analysis toolset for JavaScript and TypeScript. Josh is also the author of the O'Reilly Learning TypeScript book, a Microsoft MVP for Developer Technologies, and a co-founder of SquiggleConf, a conference for excellent web developer tooling. Find Josh on Blue Sky, Fostadon, and dot com as Joshua K. Goldberg.

• 2:17Speaker 01

  With me today is David Burns, head of Developer Advocacy and Open Source at BrowserStack. David, welcome to Software Engineering Daily.

• 2:25Speaker 02

  Thanks for having me, Josh. I'm really excited to be here.

• 2:28Speaker 01

  Yeah, we're excited to talk with you. You've got a lot of cool stuff to talk about, developer advocacy, and Selenium, and W3C, and open source. But before we get into all of that, tell us, how did you get into tech?

• 2:38Speaker 02

  So I was born and raised in South Africa, went to university there. Then, when I was at university, I was studying computer science and industrial psychology, which I know is kind of slightly weird and wonderful. But my idea then is because I couldn't make up my mind if I wanted to be in tech or a psychologist. And then moved to the UK to kind of just do a bit of traveling. Got a really cool job at a bank, which was doing kind of process reengineering. And then I was like, oh, this is kind of cool. Learned about about like process automation and then started at a startup in Southampton, which is on the South Coast.

• 3:20Speaker 02

  I was their QA hire. They'd never had QA before and they were like, we need to fix our things and we need automation. And this is kind of when CRs were up and coming, so kind of like 2006, because I've been in this industry for a while. And yeah, I started doing that. Then there was this weird thing called Selenium that was just starting out. And my wife is a musician, so she was away in the evenings with work. And so I started doing a lot of open source to support them because I was like, well, what else am I going to do in the evenings? I wasn't that much of a gamer. And started doing all of that, then got hired at Mozilla, spent a decade at Mozilla doing more open source, got into standards bodies, doing things like that.

• 4:05Speaker 02

  Because as a whole, I kind of think the internet is this most amazing thing. There's bad parts to it, but like with everything in life. And I really believed in Mozilla's mission, so I spent a decade there. And then for the last six years of being working at BrowserStack, I started up their open source program office, working with open source projects, supporting all of those, doing developer advocacy, supporting conferences, just trying to get the word out on what's the best way to develop software and test it.

• 4:37Speaker 01

  Let's go through that in order. You are a QA at a shop that previously had not done very much of that. What is that experience like? How does that feel to you?

• 4:47Speaker 02

  It was odd, especially because it was my first QA job as well. Like I'd been working with a lot of QA people at the time when I was working at the bank. And it was interesting to kind of solve the problems. And nowadays, everything is so much easier. But because like, you know, from version control where you literally had to check the file out, no one else could work on the same file. Like we were talking back in those days to how would a CI do things? How would you build out all of that? And that was interesting and I learnt a lot. And I think it helped my career a lot because like I was given a lot of freedom. And then I was just like, go figure it out for us because we don't have the time to figure it out.

• 5:29Speaker 02

  And so had to be my own SRE in the QA environments, had to build out automation frameworks around and help people do that. Introduced things like TDD because that was coming up through Extreme Programming at the time. And I was like, wow, this is actually a really good idea. Like you're thinking about testing from the start. I think we should do more of that. It was fun and I spent, I think, four years at that company. Like they're really great engineers, really great people to work with. It was like marketing software, so I also learned how to market myself because like I had to learn to be a marketer, to be a good QA, to understand their stuff. And it was, yeah, it was, it was fun. If I was to go back in there, I'd probably change everything I did,

• 6:14Speaker 02

  but it was a great learning opportunity for me.

• 6:17Speaker 01

  What is, just to quickly ask, what is one thing that comes top of mind that you would want to change from how you did things back in the day?

• 6:25Speaker 02

  So the thing that probably stands out the most is I was trying to introduce, because DSLs were the fashion of the time, right? So you would take a framework and then build out using your language of choice, like how the API would be. And I was like, oh, Fluent APIs are really good. So I'm going to build a DSL to our company. So DSL for those listening is a domain-specific language, if you don't know what that means. And so I built up this DSL, did it in a fluent way. So it was always like, do this dot, do that dot. And I built all of that and I was like, this is really cool. And then I started debugging it. And I was like, wow, Fluent APIs are literally the worst, right? Because everything's a dot.

• 7:10Speaker 02

  When you hit the debugger, you've got to go through so many layers just to get to where the real problem is. And so that would be my first thing. It's like, don't do that. I know it looks nice, but it's hard to work with.

• 7:21Speaker 01

  Agreed. I do miss the days of jQuery and Fluent everywhere and Link back and C sharp. Those were fun, but agreed. But all right, let's move on to the next item. Mozilla, you were there for, it sounds like, quite a while. What is it about the Mozilla mission that resonated with you?

• 7:37Speaker 02

  So there were two parts to it. When I was at university, I got really into open source, learning about what they were doing. And the idea of that software should be free. And if you want services around that software, you pay for that, right? But the idea is that you give out as much knowledge to the world to improve the world as a whole. Right? Like that was always my belief in open source. And Mozilla exuded that, right? And earlier, I said, kind of like, to me, the internet is the greatest invention that we have, right? It's this place where you can share knowledge, you can learn. Ideally, it should be free for everyone to do that. It doesn't necessarily always work out that way because capitalism doesn't work that way.

• 8:23Speaker 02

  But there's some really good things around it. And when I was doing my interviews, the interview process, when I joined, so when I joined, there was 350 people at Mozilla, like really, really small company. But everyone I met was probably the smartest person I'd ever met on the planet, right? Like they were just these people who were like, we've got to solve these really hard problems and at scale and make the world a better place, right? Like that was their belief. And I was like, okay, this is really a place for me. And they were all super nice people, all from around the world. I was looking to change jobs because I was driving like an hour to work to Southampton. And my wife was pregnant at the time and I was like, you know,

• 9:07Speaker 02

  I don't want to be driving like an hour there and an hour back because I'm missing out on family time. And so then by accident, I found Mozilla were hiring, went through the process and I was like, okay, this is a really cool place to go work at because the technology, the people, kind of you talking about jQuery, like when I was there, the creator of jQuery was working at Mozilla at the same time. Met Brendan Ick, the creator of JavaScript. So it was like, there's these super smart people who are willing to tell you everything and share their knowledge. And I learnt so much every day that I worked there.

• 9:41Speaker 01

  Sounds like a lovely place. What were you working on when you started?

• 9:45Speaker 02

  When I joined, I joined as a QA lead on a project called Socorro, which is the website that allows people to see when Firefox crashes, why it was crashing. So you could see stack traces, things like that. And the interesting thing is when you got, well, I think at the time there was over 500 million users. When there's a crash, even if it's a 1% crash on 500 million, that's a lot of data coming in. So it was like a lot of learning how to scale, things like that, putting it through its paces. So I did that for maybe four, five months. And then I got moved into a team called the Automation and Tools Team,

• 10:27Speaker 02

  which built a lot of the testing frameworks that Mozilla used. Cause everything had to be handcrafted, like you can't just go buy a tool off the internet type thing and it'll work. And so there was a lot of doing that. And they also managed some of the CI infrastructure there. And so I started doing that and then kind of went into, by about year three, I became an engineering manager. Cause I saw humans as a different type of problem to solve. And I really enjoyed that side of things. Like I really enjoyed the people aspect. And I think that's down to some of the industrial psychology that I was talking about before. I got to see people and I'm like, oh, this is really cool.

• 11:11Speaker 02

  And then, yeah, I spent the next six years solving various different problems from build to CI to CI visualization, scaling problems, things like that. And still also having a bit of time to work on projects like Selenium and things like that.

• 11:29Speaker 01

  That's a great segue. I'd like to dive into the automation area of work, but this is kind of a big area. You have the standards body like W3C, you have subcommittees, you have Mozilla. What would you say is the main list of things that people should know about when trying to understand how all this stuff fits together?

• 11:46Speaker 02

  As in how does standards and a company like Mozilla fit together?

• 11:51Speaker 01

  Yeah. Like, how do you get products like Firefox and open source projects like Selenium all fitting together and working together?

• 11:58Speaker 02

  So when I joined Mozilla and I've been working on Selenium, I think for about four years, by the time it became, like we started talking about standards. And there was an interesting discussion. I was at a conference in Switzerland. So for the older listeners of this, there used to be a really cool conference called the Google Test Automation Conference, GTAC, and it used to be around the world. And one year someone from Opera arrived and he's like, look what I've built. And he had built Selenium into Opera. And we're like, this is amazing. We think every company should be doing it that has a browser. And he was like, well, the way to do that is to get into standards.

• 12:43Speaker 02

  And I'm like, we have no idea what that means. Obviously like I've been doing a few things and I would read like specs and see how that related to my work. Because there'd be times where it's like, oh, we're adding this new feature, which normally came from a specification so that we could have interoperability between browsers. And I might help people create testing frameworks around that to make sure that they can test it. But there was never something that was like, here's a standard to build a testing framework into. It was not a testing framework, an automation framework into the browser. And we started doing that and it was a chap called Andreas Tolf Tolsen, who then about 18 months after that joined my team.

• 13:26Speaker 02

  And we started building out the specification, everything like that. And I was working very closely with Simon Stewart, who created Web Driver to get the standard going. He was at Facebook at the time. Once we got it really going. And yeah, we finally realized how important it was because when it came to how standards test a lot of the browser, a lot of that was very manual and very slow and very error prone. And you would get some people who would be like, oh, when I've looked at these two images, there's this half pixel that's different, must fail. And so I remember there was a story of an Internet Explorer QA who was trying to do all of that.

• 14:08Speaker 02

  And they focused on perfect versus good, right? In certain circumstances. And they spent so much time and it was like, it was kind of wasted time. And so again, for the older listeners, there used to be the acid tests for rendering. And certain browsers had a hundred percent, but other browsers just got to 98 and were like, that's good enough, right? And it was a lot of work around that to get that story out and build it out. And building a standard is actually a lot harder than it might look than just this one page you might read on a W3 or what WG website.

• 14:46Speaker 01

  Yeah. What goes into that? How does some text page get generated behind the scenes?

• 14:51Speaker 02

  So once you've got the idea for a specification, that a lot of the work that we did was trying to decide what we could standardize and what we couldn't. So for Selenium, like the easy things were, I want to type, I want, or Web Driver, I want to type, I want to click, I want to move the mouse, I want to drag drop. Those seemed relatively easy. The interesting problems came was like, when do you know that an element is actually on the page? Or how long do you have to wait? And the problem there comes down to somewhat standardizing a meaningful wait without trying to solve the halting problem.

• 15:33Speaker 02

  So the halting problem, famous Turing problem. And it's like, it's impossible to solve, right? And automation becomes so hard to figure that out. And so you're not trying to solve that one. The other one was, is something visible when you want to do it? Because when Web Driver started, Simon was like, when someone clicks on something, it needs to be visible. What does visible mean, right? And without doing pixel by pixel checks, because you can have overlays and things like that. And CSS has a Z index, but it's not really a Z index. Because like you can have other things that can overlay just by the way you lay out the DOM and the CSS and things like that.

• 16:18Speaker 02

  And it becomes very hard. And that's where a lot of the effort went into was trying to solve these slightly harder problems that people didn't need to necessarily solve. And then create a way that people could extend out Web Driver through other standards, through other specifications, so that whenever they were like building something, they go, oh, we know how to test this now. And that's how a lot of browsers now test is that they say for permissions as an example, right? Whenever you try log on, go onto a website and it's like, do you want to give this website permission to your camera and your microphone? So, you know, I'm using that as an example because we're recording. How do you test that? That becomes harder.

• 17:00Speaker 02

  And so it was like, oh, we've built out these Web Driver APIs within the specification. So now you can extend it out and start testing. And that's where a lot of effort now goes into is like just trying to make these things better. That was the Web Driver Classic specification, as we like to call it now. And a lot of the work nowadays is on Web Driver BIDI and BIDI stands for bidirectional.

• 17:24Speaker 03

  You're listening to DeepGram Flux TTS. Different voices, same model, all ready to speak.

• 17:30Speaker 04

  Flux TTS is a streaming text to speech model built for voice agents.

• 17:34Speaker 03

  Flux reads the room. It holds context across the conversation, turn after turn.

• 17:39Speaker 04

  With consistent tone, interruption handling.

• 17:41Speaker 05

  And plenty of personality.

• 17:43Speaker 03

  So your agents keep it flowing and customers can just keep talking.

• 17:47Speaker 04

  Try DeepGram Flux TTS free now until September 12th. Visit deepgram.com/keeptalking. Terms apply.

• 17:54Speaker 00

  Every visitor on your platform looks the same at first. Can you trust them or are they a threat? Fingerprint lets you answer this question right away with one API call, no model to maintain and implementation in minutes. Fingerprint recognizes bots, AI agents, VPNs, tampering, and over 100 more unique signals to give you durable risk scoring and threat detection in real time. Even as human and agentic activity blends together, you can identify suspicious traffic faster and build safer and more secure sites and apps. Over 6,000 companies rely on it every day. Join them now at fingerprint.com.

• 18:35Speaker 00

  Think about your mobile app source code. Once it hits the app store, it's out in the wild. And without the right protection, decompiling is easy for malicious actors looking to steal your IP or tamper with your software. That's where GuardSquare comes in. GuardSquare provides the highest level of mobile app security for Android and iOS applications and SDKs. Their advanced tools integrate seamlessly into your CI/CD pipeline. We're talking polymorphic, multi-layered code hardening techniques and automated runtime application self-protection, paired with mobile application security testing and real-time threat monitoring to deliver the highest level of mobile app security without compromise.

• 19:18Speaker 00

  Don't leave your hard work exposed. Secure your mobile applications today. Go to guardsquare.com to learn more.

• 19:27Speaker 01

  So just to confirm, Web Driver is the standard that describes how you might send and receive commands from browsers? Things like click, is this element visible? And then you have tooling built on top of that, like Selenium that uses that for stuff like, say, automation or accessibility testing. Is that right?

• 19:42Speaker 02

  Yeah. Selenium, Web Driver I/O, all of these things, Nightwatch JS. There's a lot of tooling out there that builds on top of it. And the only real reason that they could do it is that they could simplify how to speak to browsers very quickly and not have to worry about, oh, how am I going to speak into it? Because originally the way you, we used to speak into Firefox was through an extension built into Firefox. But when Web Driver was working with Internet Explorer, you had to do a lot of COM interop, which was very hard, especially in different languages. It's easy if you're coming from a Microsoft language, intercom, .NET and all of those, they did that fantastic.

• 20:24Speaker 02

  Java intercom, not great. Python and Ruby, even worse. Node, who knows?

• 20:32Speaker 01

  Well, it's nice to have that standardized. Thank you for working on that. Much appreciated. But it's interesting. There's kind of a through line in your career where you started off doing QA, working on the browsers, setting up tooling, and then you just worked on that tooling and you went down into the depths of it. So let's go back up towards the surface because you now work at BrowserStack. What is the relationship between, and also what is BrowserStack and then its relationship to things like Selenium or Webdriver?

• 20:55Speaker 02

  So BrowserStack is a testing platform, software as a service, and has various products from a test management tool to AI tooling around testing nowadays. It started out as being Selenium in the cloud because scaling is one of the hardest problems that people tend to have to deal with. And not everyone has that experience or knowledge, and especially in the QA space. And a lot of companies were just like, well, I'm hiring another person where I could just pay for the service. So they do that and kind of mobile. And when I was hired, one of the things that I was hired for was giving back to a lot of the communities that had made BrowserStack.

• 21:40Speaker 02

  So it was like, David, go work on Selenium still, right? Like, that's what we want to do because it's important to us as a company to maintain good relations with that. And we definitely don't want to own any of the Selenium stuff. We want to be able to keep there, same with Appium. So I have a very good relationship with the Appium folks, Webdriver, I O, do all of that. And I still get to maintain a lot of the work that I do through standards, which of course our competitors benefit from, but everyone benefits. And the idea is hopefully a rising tide lifts all ships, right? And makes everyone's life easier along the way. So I do a lot of that and then go to conferences and like, hey,

• 22:25Speaker 02

  here's this really cool thing that we've built out. Or we're doing this thing and we're giving it away because it's important to the industry and things like that.

• 22:34Speaker 01

  You reference the difficulties of joining the utopia that is open source with the real world that is capitalism. So let's say that I'm a, you know, a big company executive and I work on tooling and such that interacts with open source. How would you try to convince me that we should have someone like you who tries to lift all boats with the rising tide, including our competitors?

• 22:53Speaker 02

  I think the simplest one to an executive, because executives think of everything within risks, right? What's the risk of everyone stopping work on that project? I think it's the biggest one, right? And we've seen through various open source engineers who've got to the end of their tether and just gone, you know what? Go to hell. And then they poison their whole project, which then poisoned everything. There was one that happened, I think, kind of 2020, like just during the pandemic. I think it was LeftShift, which was a node package. And the engineer had been asking people like, hey, I've lost my job.

• 23:35Speaker 02

  You all rely on me for this thing. Could I have some money, please? And no one gave him money. And then he poisoned all his packages and then deleted all the code, right? And I think that's the big thing. So that's the worst case. The real reason why I think a lot of companies should do it is, I'm sure a lot of people got taught about it during school, but like corporate responsibility is actually a big thing. You need to look after the things that make you money as a whole, right? Supplies and things like that. And especially in the age of AI, I think supply chain attacks are going to become a lot more prevalent. So if you're in there and helping secure everything,

• 24:19Speaker 02

  yes, you give away some things, but at the end of the day, you're making sure your business is as risk free as possible. There's never going to be zero risk, but as risk free as possible.

• 24:30Speaker 01

  I would imagine having someone like you who's deeply familiar with the spec and also the politics and people behind it is probably quite useful for the company too. You know, if there's a feature you want to work or like to make sure some area is better kept.

• 24:42Speaker 02

  Yeah, definitely. And the other side of it is that if there's major problems that as a, say, our website and our product is having with, say, Google Chrome, I have enough contacts that I can just reach out to folks at Google Chrome and go, hey, we've got this bug. Could you help us? And we've reproduced it. Here we go. Could you, you know, prioritize it for us? And so there's a lot of give and take that then becomes really good. And then again, it's us giving to Chrome and going, hey, your product's got this bug. Let's get rid of it. It benefits us, but it also benefits other people.

• 25:18Speaker 01

  Sure, the Chrome people like that. Yeah.

• 25:20Speaker 02

  Yeah, exactly. Right? Every engineer doesn't want to be writing code that's full of bugs, right? And then bugs are always something that's not tested, like the bug's always there, right? It's never been tested in that way. And it's like, oh, by the way, now I can go fix it.

• 25:34Speaker 01

  Speaking of fixing and detecting bugs, let's talk about another role play. I am a website or company that has a website. Why would I care about BrowserStack or any of the other things we're talking about?

• 25:45Speaker 02

  I think part of it, so if you've got a website, how much do you care if people actually use it? If you don't care that people use your website, then like BrowserStack's not going to be any use to you, right? Like, I think that's the, like, if it's just a marketing front page, who cares? If you have a software as a service company, I think it tends to be really important that you at least have one test for every critical path. I think it's the most important way. Doesn't mean you need BrowserStack, but every critical path, once you start writing one test for every critical path, becomes important and it becomes bigger and bigger and bigger. And then there's regulatory things that also need to be put in there.

• 26:29Speaker 02

  Biggest one that's been on people's minds for years is accessibility, right? I think that's going to be really big. In the AI space, I think governance is going to be another big one that I think a lot of people are going to think, need to care about and how that interacts with everything. And so a lot of people just see the price and they don't look at how much am I, from a cognitive load, am I taking off from the company, right? So like, even if I didn't work at BrowserStack, I would always recommend people use a BrowserStack-like company, where possible, because if you're going to hire people to maintain infrastructure for that, you can't just hire one person. It has to be two people, right? Because you need to have a backup, because otherwise the bus factor is really bad.

• 27:13Speaker 02

  And then it's the, how much maintenance do they do constantly? And then you've got all this other infrastructure where a lot of these companies like BrowserStack were able to spread the costs around a little bit easier. And we take a lot of that heavy lifting. And then people don't need to care about it. They just know that it works and you've got SLAs and you've got all of that. And then life's a lot easier that way if you don't have that cognitive load on you. When you can just think, I need to test this. I need to make sure my product is high quality. High quality products lead to more sales for me.

• 27:47Speaker 01

  That's very convincing. I'd like to go through each of the three main areas you touched on and ask for some advice here because I think a lot of our listeners are perhaps getting started or have existing flows, perhaps even through BrowserStack. But it can be hard to know, what should I test? How should I test it? So do you have any tips or algorithms for I have a SaaS or I have an existing complex website? How do I know what I should test to begin with?

• 28:10Speaker 02

  So the advice I always give, and I've kind of alluded to it already, is the create one test and build out from there. And ideally, you should only have one test for one critical path. Because if you have thoughts about how you're architecting your system, you'll have other layers of protections built in from observability to telemetry to kind of everything in between, right? And you can see, and logging, you can see all these things happening in real time. But if you have that one test that I can log in and do X and that's solved, that's great. And then you try work out which of those different layers become a lot more important along the way.

• 28:52Speaker 02

  And so that when you go, oh, I need to add another test, it's like, well, this is a critical path test. I need to do this rather than, I've got 15 tests that are roughly doing the same thing. Each test is going to take like a minute, so that's 15 minutes of testing. Do you really need to wait 15 minutes? So it's about trying to balance how long things take, how critical, and then what is the risk factor within that? There is no one way, but it's about balancing all of those along the way so that you get the best out of it. It's like, if your test suite takes 24 hours to run, could you stop splitting that into multiple runs throughout the day, but it's in different sections or things like that?

• 29:35Speaker 02

  And then suddenly, the more you do in parallel, the quicker you get. And then again, coming back to the cognitive load, you don't have to go, oh, what was I doing again 24 hours ago before I found that bug? Right? And trying to do that. So it's like speed versus correctness versus risk.

• 29:52Speaker 01

  It's interesting. A lot of people refer to things like having smaller test suites or more rapid development cycles in reference to unit testing or even integration testing. And you're using very similar terminology here for the end to end browsers testing to try to make it easier for developers to use.

• 30:07Speaker 02

  What I've seen in my career is a lot of people tend to do lots of unit tests, like do the pyramid, right? Testing pyramid. So you have small tests at the bottom, medium tests in the middle and long tests at the top. And obviously it should shrink from bottom to top. I've also seen a lot of people move to a space, especially now that component testing is a thing around like React and View and all of that. Like, how do you test this one component? And they build out either a Christmas tree shape in their testing pyramid, or I've seen someone else describe it as the testing hourglass, where it kind of shrinks above and then shoots out again very quickly. You don't want that,

• 30:48Speaker 02

  which is why I think writing one test is the most important for each critical path. Because then you've got that really good one test for doing your thing, that really good one test for accessibility, one test for visual testing, one test for everything else. And suddenly you might have a hundred tests, but those hundred tests are the most important tests you need when you've got a browser, because browsers starting up are really expensive in compute time, right? And you want it to be as fast as possible, but browsers are so complex, they're effectively operating systems in themselves nowadays, right? We have Chrome OS, we used to have Firefox OS, right? Like, they're incredibly complex. So if you're constantly starting them up, you're just booting a computer

• 31:33Speaker 02

  each time, and it's a computer within a computer. So like, let's see if you can shrink things down and think about where you're testing it at a better place. I'm a big advocate of trying to push lower down the pyramid where possible, but I do think you still need those end to end tests.

• 31:49Speaker 01

  Excellent. You've mentioned now a couple of higher level tests. Actually, there's accessibility and then perhaps visual regression. How would you advocate for accessibility testing for a business that has customers and perhaps contracts with other businesses or governments?

• 32:02Speaker 02

  I think a lot of people forget that a lot of accessibility products started out as accessibility tools. So if you're a developer and you work in your IDE and free AI, a lot of the move was like, how much work can you do without touching your mouse? And if you focus on getting that correct, that's an accessibility test, because not everyone can use a mouse. Another great example that I'm sure a lot of people use is if you're driving and you get a text message and your phone reads out that message and you can reply to it. A lot of that was built around accessibility tooling, because it was around people who couldn't necessarily use the phone, but they wanted to speak into

• 32:45Speaker 02

  it and do things like that. And so there's a lot of these productivity wins that a lot of people have. So it's like, how quickly can I get to what I need? So if it's a travel website, can I just tab to where I want to do? Because I just want to find the thing the quickest, right? Especially in today's attention economy, you don't have a lot of attention. So like, how can you make those things better? And just by making that productivity better, you're solving accessibility at the same time, because you're making it a lot easier and you're thinking about those things. There's also, I think, a lot of work happening in the automation space around like screen readers and things like that. So there's a specification that's kind of trying to be built at the moment, which is called the

• 33:26Speaker 02

  accessibility test driver. So kind of take on web driver, and allows you to automate screen readers to navigate through a website. And so now you go, oh, I've got this really cool thing that works and solves. And so focusing on those things, accessibility becomes a natural thing, where it's like, the flow is better, things like that. And I think that's where people need to focus on, rather than, oh, I've spent all this money for this law and it's not really benefiting us. It's like, your thinking is kind of backwards on that.

• 34:00Speaker 01

  To use an analogy, what you're describing is a lot of carrot rather than stick, where these are good reasons why one would want accessibility. And traditionally, there are also a lot of sticks here, you know, lawsuits and such that would force a company to be accessible. Do you see there being a shift in the balance between how people kind of advocate within that?

• 34:18Speaker 02

  Hopefully, a lot of companies will always only focus on the stick, right? And there's not much you can do about that. But if you can get them to focus on the carrots a lot of the time, and go, oh, we've added this feature, and this is how much benefit it has over our competitors, without necessarily mentioning accessibility, but it's got all these accessibility tools. I think there's huge wins there. The other side of it is like, accessibility counts for, I don't know, how many different percent in different countries, right? And if you don't have an accessible website or things like that, or app or whatever, are you happy with not caring about the sales from those types of people, right? Because their money is just as good as mine. So do you not want that?

• 35:03Speaker 02

  And the same argument can be had for people going, oh, we don't need to test in, we'll test only in Chrome nowadays. It's like, okay, cool. But you know, you've got all these other companies that are working in the space, like Apple with Safari, Mozilla with Firefox. Do you not care about that wedge ever coming to being sold there? And then that changes the narrative slightly again, right? So it's like, a lot of the time is trying to show people the benefits, rather than going, oh, there's that stick over there and we don't ever want to be beaten by it, rather than, if we do this, this is, you know, it's the right thing to do, but it adds all these really cool features.

• 35:41Speaker 01

  That's a good way of putting it. I also want to dive in a little bit. You mentioned there's this new accessibility test driver specification. I've used a lot of Axe Core as a library for accessibility testing. So like, I guess for our listeners, what is Axe Core? What is this driver specification? What's happening in the space?

• 35:58Speaker 02

  Okay, so Axe Core is just like a tool that works through a DOM and goes, these are the accessibility problems that are there. There are other competing tools out there, like BrowserStack has a tool called Spectre, which does the same thing, but kind of does it in a slightly different way. Because with a lot of tools, they read through the DOM and then try infer certain things. AC Driver Accessibility Test Driver is very different in that it's trying to drive a screen reader to then drive the browser. So it's trying to find these hooks through to somewhere else to do it. And so that if you press tab through the screen reader, it would give you some, like where you are and describe

• 36:43Speaker 02

  it to you. And is that a standard way of describing things at the time? Because with screen readers, accessibility people will feel this frustration, is that like every operating system, screen reader acts very differently. So a screen reader on Linux is very different to what would happen on Mac to Windows. And then you have different competing companies on each of those platforms who do different things, right? And it's the same problem that people have when doing automation on mobiles with Appium, especially other tools too. But the way you do things on Apple is very different to the way you do things on Android, because there's no standard and like, there's no want for a standard, because they're like, oh, we're kind of monopolies, it's good enough, right?

• 37:28Speaker 02

  We don't need to care about that problem. It creates other things. But with accessibility, I think there, they're like, well, we should make things better, because then we can figure out the way to do things and have a standard way. It then improves interoperability between Chrome and Firefox and Safari when doing these things, when you can go on Linux, we've done this navigation through a website, it looked like this. And then we did it on Mac, it would look like this. And this is the same thing that a person might want or need.

• 38:02Speaker 01

  That makes a lot of sense. It's interesting how a lot of these kind of shared common concerns keep getting discovered in the wild and user land and then turned into a spec. First, you had the actual talking to the browser. Now you have these slightly higher level concerns. What do you think about this space's evolution? Is it exciting to see stuff continuously get specked out?

• 38:20Speaker 02

  It is. And this one frustration that I think a lot of your listeners will have is that whenever something's going through a specification, it's so slow to actually get specified, right? Like, when I started work, JavaScript was just a mess across the board, right? Between different browsers and you needed to do all these different things and you would figure it out through callback hell and like you'd go through it. And then a lot of the ECMAScript standardization came in. I was like, oh, this is really cool, right? When we started working on this, and then they started creating test suites that you could run through everything. But the slowness made it correct, right? Because if

• 39:01Speaker 02

  you make things fast, you cause other problems. And we've seen that in like the spec world. So there was a specification, I can't think of the name of it at the moment, which is used for people to send messages between browsers. So if you want to make a phone call or things like that through a browser, you would do it through this. And it's on its third iteration, and the third iteration is going a lot slower, but it's being more correct. Because the first two iterations, it just caused so many different problems between like, oh, you couldn't make a call from like, say Firefox to Firefox or Chrome to Chrome, but you can never do Firefox to Chrome. And that caused problems. And so

• 39:44Speaker 02

  that's where it becomes really important to get that correct. It's the same with the web driver specification, right? There are competing tools out there to Selenium, like Playwright or Cypress, and they were never really that involved in the specification part. But when we've been working on things, we're trying to make it that, oh, you can always extend it out and then go, oh, we're going to add this feature to the browser, automatically gets a web driver test. And so you can know that whatever you get from different browsers, and that becomes really important in the mobile space a lot more. Because like Chrome on Android, Safari on iOS, you need to think about how those things interrupt with each other along the way and get the best out for people.

• 40:27Speaker 01

  That's really exciting. I'd like to shift forward for our last kind of technical section and talk about the future a little bit. Obviously, AI is the biggest elephant in the room. But in general, where do you see the space going? You know, what's BrowserStack up to? What's W3C going on? What's what's happening next?

• 40:43Speaker 02

  So BrowserStack's still moving forward, still focusing on testing as a platform. And I think that's going to be really important. I think in the age of AI, I think testing is going to become far more important. I did a talk earlier in the year around vibe coding and vibe testing. And vibe coding is this like term where you just write a prompt and hopefully you get the right thing, right? Like you kind of vibe it out. And then people are also doing that for testing. The problem then comes is that people will vibe their production database into the ground and like it just gets thrown away. We've seen numerous stories and it's not actually getting any better. I think we saw stories about like Replit and things like

• 41:28Speaker 02

  that last year, who then have putting guards and saves things into that. And then people were like, oh, I don't need Replit anymore. I'll just use Claude. And then someone put in something, didn't give it guardrails. And I was like, oh, my production database is now deleted. Great. And so I think the AI space is helpful, but I think at the same time, we still need people who understand what AI is generating to be able to like give it the guardrails, fix it, solve those problems. Because there's a lot of context that when you and I are talking, that our lived experience will be adding to whatever I'm saying. And AI doesn't necessarily have that kind of thing. But it's also really good at doing quick prototypes

• 42:12Speaker 02

  or transformations that are really important. So moving data from one place to another, you can go, oh, just do this. It's really cool. And I really do like AI. I just think it's been democratized too quickly to everyone. And not that they shouldn't have it, but it's like, there's a lot of security problems that then tend to kind of crop up.

• 42:33Speaker 01

  Do you think that as people get more familiar with AI, these problems will become lesser? Or do you think that as AI becomes more prolific, the spread of them will become greater than that?

• 42:43Speaker 02

  I think the latter. I think the more we democratize it, the worse the scenarios will get. When I was working on browsers, one of the things that was always driven into us was like, always think about these lists of problems that come up. So a lot of browsers are built in native languages like C, C++ Rust, things like that. Rust, not so much of an issue these days. But when we're C and C++ it's like, oh, you know, don't try access a variable after you've told it to go to the garbage collector. Right? And unless you're kind of telling people to do that when you're building out things, or, you know, is it actually checking that the person is authenticated? Are we by accident putting API

• 43:26Speaker 02

  keys into our code when they shouldn't be? Or, you know, they should be properly obfuscated or whatever. A lot of that is learned knowledge that you and I might have from building software. But if people have never built software and had that drummed into them from the start, I think that's really dangerous. And I think while I like AI, I kind of feel like junior engineers should be using it a lot less, more of a backup to like, review their stuff and go, hey, did you think about this problem, that problem, right? Mostly so that they can have that knowledge drummed into them from the start. But then go do what you need. When it comes to prototypes, I do think everyone should have an opportunity to build things

• 44:07Speaker 02

  out. I've got this idea, let's take it, oh, okay, this is really cool. But then it needs to be rewritten, or at least go through security checks to make sure that if the prototype goes to production, it's safe. Where prototypes, say 10, 15 years ago that you were building, you had rough ideas where the problems were from the start. And you could quickly fix those up before it went into production.

• 44:30Speaker 01

  There's an interesting parallel here with Stack Overflow, where an experienced developer can go on and copy and paste whatever answer and know that it's good. But you're kind of depriving yourself of the learning if you do that before you know what's actually happening.

• 44:43Speaker 02

  Yeah, agreed. Definitely. And that's what I mean, right? I view AI as, for those who've done a lot of .NET and things like that in the past, there used to be a really cool tool called Resharper. I don't know if it's still a tool, or if it's just built into everything that JetBrands makes. But it was so good at what it was doing and being able to kind of give you the good templates and things like that, that you can just build out stuff really quickly. And I think that's where AI is. And it's really good at that. It's also good at giving you lots of information, but then you still need to have your knowledge to go, what is correct or not correct, right? AI will hallucinate. That's not a maybe hallucinate, it will hallucinate. It's how often it hallucinates is becomes the interesting thing. But you can

• 45:27Speaker 02

  never tell the difference unless you've gone back and reviewed it.

• 45:31Speaker 01

  Absolutely. Shifting gears a bit back to you, what are you excited to work on in the near and in far future? What sort of projects do you have going on?

• 45:39Speaker 02

  So kind of as a little pet project, I'm trying to get more people looking at observability. I think that's really important in today's age. So observability allows people to create traces through applications. And people have been doing them for about eight years, nine years with the open telemetry stuff. The difference is, I don't think a lot of people put it into their tests and then be able to see a test go from, I've hit the website all the way through their application to know when something fails. I think that's really cool. The other thing that I've been doing a lot of reading up on is CD events.

• 46:18Speaker 02

  So continuous deployment or continuous delivery events, which allows you to kind of build out these event-driven things into your CI and pipeline. So you can always track where something could be. There's a lot of good work going there. It's still kind of early stages, but there's interesting things happening in that space.

• 46:38Speaker 01

  That's an interesting one. So you're looking at connecting this kind of a standard, say, end-to-end browser tests with much more rich tooling with observability. Does this make it easier to debug or understand when failures happen?

• 46:50Speaker 02

  Yes, exactly that. Because observability allows you to create traces, right? And so if you go, here's my trace key, and I want you to use it all the way through. Some of those things get picked up in how you develop and pass things through. So I've got a whole bunch of talks coming out towards the end of this year, where I'm showcasing some of the work that I'm doing. And I think it's got some really important things, because for years, a lot of people would just focus on reading logs. And you have to know what connection that has, right? But you could always then, like when Open Telemetry came out and people were building it, you could go, oh, I could see you go through here. But you never knew exactly how it started. You just saw the data coming into the system.

• 47:34Speaker 02

  What if you could add it to a component test or a browser test, right? Or a mobile test? And you go all the way through. And then that creates some really interesting ways of creating, because then if you've got, you see a failure and you go, oh, I have this data. Let's see if I can replicate it, how it came in. And if you fill in the form with the same data, and it worked that time, you go, oh, okay, maybe someone's doing it a different way. And now you can try figure out how that person was doing it, rather than going, I don't know how to start this, let me figure it out. And then kind of spending time. So it's about trying to frontload some of that information into your test and then following it all the way through.

• 48:14Speaker 01

  So how long until you start contributing to the open telemetry specifications?

• 48:19Speaker 02

  I don't think I need to. This is all stuff that's there, right? Like, I've built out stuff that shows it. So the only thing that might be needed is how people start up their tests and giving them just a basic thing for this is like, because you need to manipulate the initial network connections going into the server. But you're not manipulating them by much, you're just adding a few headers, right? You're not actually manipulating and then following it through. So I don't need to do any changes to the specification. Like, it's all built in. It's just like, here's this thing that no one's really been talking about. So I did a podcast with Marie Cruz, and she works for Grafana. And we were talking about it. And she was like, oh, wow, I wish more people were doing these things where

• 49:03Speaker 02

  it's like, it's already there, but they don't know how to get started.

• 49:06Speaker 01

  Really exciting. For our last few minutes together, I want to completely shift gears. I like to end each interview with something non-technical that has nothing to do with work. David, what is a fun fact from history you'd like to share with the audience?

• 49:19Speaker 02

  So I grew up in South Africa, as I mentioned at the beginning. And so what a lot of things that I learned growing up was around how South Africa as a country was founded. And a lot of the history around the British being in South Africa. And so I was always fascinated because a lot of history from South Africa is being generated through word of mouth over the years, right? And then people have to go re-review it. But Shaka Zulu, well-known Zulu king for South Africa, one of the things that he did was he trained his soldiers to learn to run on thorns, because they were barefoot, running through kind of the African

• 50:01Speaker 02

  countryside. Mostly because if they were doing that, strengthen the soles of their feet, but then also they wouldn't think of the pain there when they knew they had to be somewhere. And so I always think of that as like, an interesting thing of, it wasn't about building shoes or whatever. It was just like, how can I strengthen this thing? So I can focus on other bits and pieces later.

• 50:22Speaker 01

  That's wonderful. That's fascinating. I had no idea about that. I have to ask, is there any analogy you can draw from that to testing or observability in production?

• 50:32Speaker 02

  I think the main thing is just trying something new and then trying to harden it up, right? And it's more, maybe not observability, but it's around security. Because I think we should always be focusing on hardening our security around things. And you need to have these layers. And so it's about building up those things. And some of it is through making mistakes. But if we make those mistakes in a safe environment, like he was doing with his training of his warriors, it becomes far more interesting that way that we can like learn and then evolve and make things better.

• 51:08Speaker 01

  Well, that's a beautiful way to end an interview. David, if folks wanted to learn more about you and BrowserStack and all the spec work you're doing, where do you direct them on the internet?

• 51:17Speaker 02

  So I do a lot of things in the community. So browserstack.com/community. A lot of things in there that I do from Discord to my podcast to other things like that. I have my own blog, which is theautomatetester.co.uk. I have a newsletter on LinkedIn where I share tidbits and interesting things that I think people need to do. And it's called Automation Isn't Scary. So hopefully, kind of puts people's minds at ease when working in it. And people can just message me. I'm on various Slacks and Discords. So you can always just message me in the open and I will try help you where I can.

• 51:58Speaker 01

  Well, excellent. David, thank you again so much for coming on the show. This was great. We talked about your history, Selenium, Web Driver, testing. This has been wonderful. So for Software Engineering Daily, I'm Josh Goldberg. Thank you, everyone, for listening. Have a great day. Bye.

• 52:11Speaker 02

  Bye.