{"id":1759,"date":"2019-08-14T11:46:16","date_gmt":"2019-08-14T15:46:16","guid":{"rendered":"http:\/\/blog3.canny.io\/wordpress\/?p=1759"},"modified":"2024-04-17T20:05:18","modified_gmt":"2024-04-18T00:05:18","slug":"beta-testing-with-user-base","status":"publish","type":"post","link":"https:\/\/canny.io\/blog\/beta-testing-with-user-base\/","title":{"rendered":"Beta testing: how to choose testers from your user base"},"content":{"rendered":"<p>Beta testing is a crucial part of every new feature release.<\/p>\n<p>Even if you\u2019ve done a good job building and designing it, finding every issue on your own is hard.<\/p>\n<p>There\u2019s also a good chance you don\u2019t dogfood your own product as much as you think you do.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"gif size-full wp-image-1766 aligncenter\" src=\"https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/giphy-14.gif\" alt=\"\" width=\"500\" height=\"281\" \/><\/p>\n<p>Having an extra pair (or a few) of eyes helps cover an and all various use cases.<\/p>\n<p>This is why it\u2019s important to let a selected few people play around with a feature before releasing it into the wild.<\/p>\n<h2>Why selection is crucial in beta testing<\/h2>\n<p>If you&#8217;re an established business with existing customers, you have a goldmine of <a href=\"https:\/\/canny.io\/blog\/finding-beta-testers-saas\/\">beta testers<\/a> right at your fingertips.<\/p>\n<p>Your existing users are more invested in improving it. They use your product or service for their own benefit, and therefore genuinely want it to be better.<\/p>\n<p>However, the specific group of testers should be very carefully selected.<\/p>\n<p>You would think an open beta test would create much more feedback.<\/p>\n<p>However:<\/p>\n<ul>\n<li>Open betas are harder to manage (they tend to include lots of people), especially for small teams.<\/li>\n<li>People often join out of curiosity and don\u2019t offer much value (but that doesn\u2019t mean you don\u2019t have to manage them).<\/li>\n<li>People outside of your target group can join (their feedback isn&#8217;t the best for actually improving).<\/li>\n<li>If anyone can join, anyone can see a less-than-perfect version of the new feature.<\/li>\n<\/ul>\n<p>So, even if an open beta creates \u201cmore feedback\u201d in general, it doesn\u2019t mean it\u2019s <em>useful<\/em> feedback.<\/p>\n<p>The feedback you want from beta testing is actionable and targeted.<\/p>\n<p>There are customers that are easy to get feedback from and who are qualified and able to do it. And then there are customers who aren\u2019t.<\/p>\n<p>So, how do you choose the best beta testers?<\/p>\n<h2>Choose beta testers only from your target audience<\/h2>\n<p>This is probably the most important thing to keep in mind.<\/p>\n<p>Your beta testers should definitely be a part of your core target user base. If your target audience is \u201ceveryone\u201d, then <a href=\"https:\/\/blog.hootsuite.com\/target-market\/\">you have bigger problems than beta testing<\/a>.<\/p>\n<p>Your target audience are the people and\/or companies you\u2019re building the product <em>for<\/em> to begin with. This is why getting their feedback first is important.<\/p>\n<p>For example, Canny is primarily built for SMB-s and SaaS companies. They are our target market. They&#8217;re the companies we try to benefit most with our product (and, let\u2019s be honest, the ones who generate most of our RR).<\/p>\n<p>If we asked for beta feedback from our <a href=\"https:\/\/canny.io\/blog\/saying-no-big-customers\/\">corporate clients<\/a>, the results would be very different. Their needs and wants are very different. It would skew our results on what matters most.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"gif size-full wp-image-1760 aligncenter\" src=\"https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/giphy-13.gif\" alt=\"\" width=\"461\" height=\"286\" \/><\/p>\n<p>This doesn\u2019t mean we don\u2019t <em>care<\/em> about them. We just don\u2019t prioritize their feedback when asking for feedback on a feature meant for our target market.<\/p>\n<p>Every company has edge case customers or users. They\u2019re fine to have if they\u2019re happy with their arrangement and not making demands.<\/p>\n<p>However, they\u2019re not the greatest people to ask for advice for when testing a new feature. They\u2019re not the ones you\u2019re building the product for.<\/p>\n<h2>Choose people who have asked for the feature<\/h2>\n<p>If you track which users have requested which features, you\u2019re in luck.<\/p>\n<p>Who\u2019s a better candidate for testing a feature than the people who have specifically asked for it?<\/p>\n<p>If they asked for it, it means it\u2019s supposed to solve a problem for them.<\/p>\n<p>With their help, you can see if you\u2019ve solved the problem right for the most important people behind it, or not.<\/p>\n<p>Not only will they be stoked that the feature is coming, but they\u2019ll be super happy to give you their input. Since they needed it and will definitely be using it, they\u2019ll be invested in improving it.<\/p>\n<p>If you\u2019re using Canny, you can just get a list of people who have upvoted or commented on a feature request.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"border alignnone wp-image-1761 size-large\" src=\"https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/Google-Chrome_2019-08-01-13-59-59@2x-1024x983.png\" alt=\"Invite people who asked for a feature to beta testing\" width=\"1024\" height=\"983\" \/><\/p>\n<p>You\u2019re one comment or message away from letting them know they can hop into beta if they\u2019d like.<\/p>\n<h2>Choose people who have given good constructive feedback before<\/h2>\n<p>Some people genuinely love giving feedback.<\/p>\n<p>We\u2019re not talking about bug reports\u2014bugs are annoyances that disrupt people\u2019s work. They have to let you know about it so you can fix it.<\/p>\n<p>Reaching out about a bug doesn\u2019t mean that someone is interested in getting involved. It\u2019s a chore.<\/p>\n<p>We\u2019re talking about people who have reached out with ideas, requests, or general feedback before. The people whose thoughts you have found to be genuinely valuable and interesting.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"border aligncenter wp-image-1762 size-medium\" src=\"https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/Google-Chrome_2019-08-06-16-23-10@2x-255x300.avif\" alt=\"\" width=\"255\" height=\"300\" srcset=\"https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/Google-Chrome_2019-08-06-16-23-10@2x-255x300.avif 255w, https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/Google-Chrome_2019-08-06-16-23-10@2x-102x120.avif 102w, https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/Google-Chrome_2019-08-06-16-23-10@2x.avif 760w\" sizes=\"auto, (max-width: 255px) 100vw, 255px\" \/><\/p>\n<p>This also includes people who are generally a big fan of your company, team or product. They care, so they&#8217;ll be more attentive.<\/p>\n<p>You can start flagging certain customers as good future beta testers, or adding them to a list.<\/p>\n<p>This way, you have a good starting point already when choosing testers for your next feature.<\/p>\n<h2>Choose beta testers from people who use your product a lot<\/h2>\n<p>Most SaaS products have \u201czombie\u201d customers. These are the users who are signed up, but use the product very little\u2026<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"border alignnone wp-image-1763 size-large\" src=\"https:\/\/canny.io\/blog\/wp-content\/uploads\/2019\/08\/Google-Chrome_2019-08-06-16-10-58@2x-1024x561.png\" alt=\"Choose frequent users for beta testing\" width=\"1024\" height=\"561\" \/><\/p>\n<p>\u2026if at all.<\/p>\n<p>Even if they\u2019re willing to beta test a new feature, their feedback isn\u2019t exactly the most constructive you can get.<\/p>\n<p>The problem is that if they don\u2019t use the product a lot to begin with, they\u2019re not aware of its full capabilities. They most likely don\u2019t even <em>need<\/em> the new feature.<\/p>\n<p>They\u2019re clearly not particularly invested in your company or product. This means that their feedback (if there even is any) will most likely be superficial and not very useful.<\/p>\n<p>It\u2019ll also most likely take way longer and be way harder to engage with them. If they don\u2019t engage with your product a lot, why would they engage with <em>you<\/em> a lot?<\/p>\n<p>It\u2019s more crucial to get it right for people who rely on your success and frequently interact with your product<\/p>\n<h2>Make sure they have <em>time<\/em> for beta testing<\/h2>\n<p>Now that you have a list of users based on these \u201crequirements\u201d, make sure that they actually have time for it.<\/p>\n<p>Even if they\u2019re the perfect candidate for being a beta tester and are willing to do it, it doesn\u2019t mean they <em>can<\/em>.<\/p>\n<p>Beta testing (especially the first rounds of it) can be pretty time consuming.<\/p>\n<p>People have stuff going on that might mean they\u2019re too busy or unable to give you constructive feedback.<\/p>\n<p>So, before you commit to your list of testers, make sure to let them know exactly the \u201cscope\u201d of what you expect from them.<\/p>\n<p>If they don\u2019t think they have the resources or time to do it currently, they can opt out.<\/p>\n<p>This way, you won\u2019t be disappointed if you don\u2019t get anything useful from them, and they won\u2019t feel bad about it.<\/p>\n<h2>Not all users are created equal for beta testing<\/h2>\n<p>As a baseline, you should always try to care for your customers equally.<\/p>\n<p>However, beta testing is a lot different from your everyday operations. You\u2019re building something new, and you want it to be awesome for everyone who will be using it in the future.<\/p>\n<p>And, you don\u2019t want to take eons to get it to that point. This means that your goal with beta testing should be efficiency.<\/p>\n<p>Whereas most users are efficient at being your customers, they might not be efficient at giving feedback.<\/p>\n<p>Pick and choose the people who are most qualified, excited, able, and constructive.<\/p>\n<p>You can move your polishing process on faster, and get your new feature to its optimal capacity ASAP.<\/p>\n<p>The best way to gather &amp; manage user feedback is to use Canny. <a href=\"https:\/\/canny.io\/register\">Try it for free!<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>There are customers that are easy to get feedback from and who are qualified for beta testing. And then there are customers who aren\u2019t. How do you choose?<\/p>\n","protected":false},"author":6,"featured_media":1764,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","_lmt_disableupdate":"","_lmt_disable":"","footnotes":""},"categories":[8,7],"tags":[],"class_list":["post-1759","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-customer-feedback","category-product-management"],"aioseo_notices":[],"modified_by":"Maria Vasserman","_links":{"self":[{"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/posts\/1759","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/comments?post=1759"}],"version-history":[{"count":8,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/posts\/1759\/revisions"}],"predecessor-version":[{"id":6613,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/posts\/1759\/revisions\/6613"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/media\/1764"}],"wp:attachment":[{"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/media?parent=1759"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/categories?post=1759"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/canny.io\/blog\/wp-json\/wp\/v2\/tags?post=1759"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}