How to beta test a Microsoft Store app

How the options differ

Set up your testers first

Private audience and package flights both use a known user group. In Partner Center, open Engage, then Customer groups, and select Create new group. Give it a name, keep Known user group selected and add between 1 and 10,000 email addresses, or import them from a .csv file.

• Each address has to belong to a personal Microsoft account. Microsoft says work or school accounts can't view or get a private audience app, even if your Partner Center account uses Microsoft Entra ID.
• Membership changes can take up to 30 minutes, and you don't need a new submission for them to apply.
• You can't delete or rename a group once it exists, but you can edit who's in it at any time.

Private audience: test a new app before launch

On the Pricing and availability page, under Visibility, set Audience to Private audience and pick at least one known user group. After the submission is published, those people can see the listing and download the app when they're signed in with the matching Microsoft account. Nobody else can, even if they type in the listing URL.

• Give testers the private link Partner Center shows when you choose Private audience. It's also on the App identity page. The regular Store URL won't work for them.
• You can only choose Private audience while the app has never been published publicly. Once a submission goes out with Public audience, you can't switch back.
• Updates you publish reach the group like any normal update. Price still applies, so set it to Free or hand out promotional codes if you don't want testers to pay.
• Reviews from your private audience show up in your Reviews report but never on the public listing.
• To launch, tick Make this product public on with a date, or make a new submission that switches to Public audience. Microsoft notes that this can mean another round of certification.

If you'd rather not use groups, you can also publish with Make this product available but not discoverable in the Store and share the direct link or promotional codes. Microsoft points out that anyone with the link can still see the listing, so use Private audience if it has to stay confidential.

Package flights: test updates on a live app

A package flight sends a different set of packages to the people in its groups. Everyone else keeps getting the packages from your regular submission, and the listing is the same for all of them.

1. 1

   Create the flight

   Once your app has a published submission, the App overview page shows a Package flights section. Select New package flight, give it a name and pick at least one group. You can't change those later without deleting the flight and making a new one.

2. 2

   Add packages

   Upload new packages, or reuse ones from a published submission or another flight. Include packages for the same device families as your regular submission, because flight members only get packages from their flight.

3. 3

   Submit

   The flight goes through certification like any submission. Microsoft lets some Windows App Certification Kit failures pass with notes while a package is only in a flight, but you have to fix them before a general release.

4. 4

   Ship it to everyone

   When the build is ready, make a new regular submission and copy the packages from the flight on the Packages page, so you don't upload them again.

• With more than one flight you rank them. A tester in several groups gets the highest ranked flight, even if a lower one has a higher version. Your regular submission is always ranked lowest.
• On desktop, testers get flight updates automatically. On Xbox they have to check for updates by hand.
• Ratings and reviews from flight packages aren't shown to other customers, but you see them in your Reviews report.
• Deleting a flight can take up to 30 minutes. Its users move to whichever package has a higher version, now or when one comes out.

Gradual package rollout: release an update to a percentage

On the Packages page of an update, tick Roll out update gradually after this submission is published and enter a percentage, such as 5. After certification, Microsoft sends the new packages to that share of existing customers, picked at random. You can drag the percentage up on the App overview page without making a new submission. The details are in Microsoft's gradual package rollout docs.

• The rollout only affects packages. Every customer sees the listing from your latest submission right away, so don't describe new features in What's new that most people don't have yet.
• Customers on Windows versions older than build 10586 don't get the update until you finalize, and setting 100% isn't the same as finalizing.
• You can't start a new submission until the rollout is complete. You either select Finalize package rollout to give everyone the update, or Halt package rollout to stop it.
• Halting doesn't roll anyone back. People who already got the new packages keep them, and you reach them with a new submission that has a higher version.
• You can combine rollout with a package flight to release to part of one flight group.

Which one should you use?

• Your app isn't public yet. Use Private audience. It's the only option that hides the listing from everyone else.
• Your app is live and you want a beta channel. Use a package flight for a fixed group of people who've agreed to test.
• The update is done and you want to limit the damage of a bug you didn't catch. Use gradual rollout, watch the health and usage reports, then finalize or halt.
• You only want control over when an approved update goes live. Use a publishing hold. It doesn't test anything, but it lets you line up the release with your own launch. The certification time guide covers the options.

Whichever you pick, the package rules are the same as for any update. The version has to go up, and the Name and Publisher have to match your product identity, as covered in how to update an MSIX app.

What StoreFast does and doesn't do here

StoreFast publishes your regular update, the one all your customers get. It doesn't create package flights, and gradual rollout isn't available in StoreFast yet. If you need either one, set it up in Partner Center. StoreFast doesn't change your audience or discoverability settings, so a private audience you set in Partner Center stays as it is.

If a gradual rollout is still running in Partner Center, finalize or halt it before your next update, since Microsoft won't take a new submission until you do.

What StoreFast does have is Hold until I publish. Turn it on before you submit and Microsoft certifies the update and then waits. Your app's page shows Waiting for you, with a link to Publish now in Partner Center when you're ready.

Questions

How do I beta test an app before it’s in the Microsoft Store?

Set Audience to Private audience on the Pricing and availability page of your first submission and add your testers to a known user group. Only those people can see the listing or download the app, and only through the private link Partner Center gives you. This works only while the app has never been published to a public audience.

What is a package flight in the Microsoft Store?

A package flight sends different packages to the people in a known user group while everyone else keeps getting your regular packages. The Store listing is the same for everyone. You can create one after your app has a published submission, from the Package flights section of the App overview page.

How many testers can I add?

A known user group holds between 1 and 10,000 email addresses. Each one has to belong to a personal Microsoft account, because work and school accounts can’t get a private audience app.

Can I roll an update back if the gradual rollout goes wrong?

Not for the people who already have it. Halting a rollout stops the new packages from going to anyone else, but customers who got them keep them. To fix their copy you publish a new submission with a higher version.

Can I take an app away from a tester?

Microsoft says you can't revoke access once a tester has downloaded the app. If you remove someone from a private audience group they keep the app but stop getting updates.

Does StoreFast support package flights or gradual rollout?

Not yet. StoreFast publishes your regular update for everyone. Gradual rollout is on our list but isn't available today, and flights are set up in Partner Center. What StoreFast does have is Hold until I publish, which lets an update pass certification and then wait for you.