When I was making my first app, I wasn't in big need to run a beta test before release. But when my sales went down after the gamers spot a crash bug that ws painful.
But I lurked through every doc and actually I don't understand how to run a beta test of the game before release? Testing on 5-10 dev iphones is not a solution, cause 10 people can't catch all bugs. How to test a game with, let's say 500-1000 users? Or those guys in apple think that buggy and undeveloped software (as they release) is ok for everyone?
Well it depends on who your testers are. If you have 10 testers who are good at testing, they can find any bug you have. If you have 10 people who just like playing games, don't expect much.
I've tested many apps over the year at work and on my own. I have a nack for finding things the application is not suppose to do. So I can conduct pretty good testing myself. But I also do an ad-hoc distribution to several other friends who I know can test an app and they give me good feedback as well.
As far as I know, most development accounts can only distribute to 100 devices total. So with your 5 devices, that's 95 left. So you could have 95 more testers.
When I was making my first app, I wasn't in big need to run a beta test before release. But when my sales went down after the gamers spot a crash bug that ws painful.
But I lurked through every doc and actually I don't understand how to run a beta test of the game before release? Testing on 5-10 dev iphones is not a solution, cause 10 people can't catch all bugs. How to test a game with, let's say 500-1000 users? Or those guys in apple think that buggy and undeveloped software (as they release) is ok for everyone?
How do you test your apps before releasing them?
If you do an adhoc distribution, you can have up to 100 beta testers.
I find it difficult to find that many beta testers.
I think it's also possible(can someone confirm?) to have the app approved on the app store but not released yet, in that case you can send promo codes for more users to try it out before release date.
I think it's also possible(can someone confirm?) to have the app approved on the app store but not released yet, in that case you can send promo codes for more users to try it out before release date.
I could be wrong, but don't think this is possible.
Well, i've been a beta tester for years for a one of the larger Mac game distributers and they don't have 500 beta testers.
They have a handful of good/decent testers that know that the application will crash and will actually write a bugreport on it.
For these users its always a trade-off between getting your hands on the latest/greatest software with the knowledge that you can/will loose all your progress in your game if you just happen to find that nice bug
I agree with roiegat that if you have a handful of good testers you really can get a long way in finding most of your bugs. Keep in mind there is NO bugfree code, (probably the space shuttle is the only exception)
Probably the best steps are:
Do regression testing with every minor release.
This should be part of your own development cycle.
Find 5 good power users (preferably fellow developers). Good for finding technical bugs or let them do performance tests.
Find 5 non technical iphone users (your mom/aunt/etc). Good for banging at all the buttons and trying things technical ("logically") minded user wouldn't think of.
500 users is s problem? Are you kidding me? I'd have 5k with ease. How do your 10 people suppose to find a crash bug that occurs in 0.001%? Ho can they test all features in all random ways?
You will rely on a test run by 10 people before releasing to 100000? To 1000000?
Cmon guys, this is business, not games...
P.S. Guys that are hired to test apps before release called QA, but did you see any online game out jsut relying on QA team, without public beta test?
500 users is s problem? Are you kidding me? I'd have 5k with ease. How do your 10 people suppose to find a crash bug that occurs in 0.001%? Ho can they test all features in all random ways?
You will rely on a test run by 10 people before releasing to 100000? To 1000000?
Cmon guys, this is business, not games...
P.S. Guys that are hired to test apps before release called QA, but did you see any online game out jsut relying on QA team, without public beta test?
Are you living under a rock?
You don't want to release until you find all crashes with an 0.001% occurence??
I mean... good for you, that's perfectionism.
But, ... not sure who is living under a rock...
You don't want to release until you find all crashes with an 0.001% occurence??
I mean... good for you, that's perfectionism.
But, ... not sure who is living under a rock...
Also these 100 UDIDs you are talking about are given FOR A YEAR! SO I will have to carry on these testers all the time all the same without possibility to change them cause I don't have ahough UDIDs?
I don't wonder any more why Blizzard does not develop for iphone...
Steve Jobs, do you hear mortals? Helloo? C'mon, dude...
So says the guy who didn't feel the need to do a beta test before releasing...
I do not assume a 10-people test a BETA TEST.
But I could not get needed amount of testers just cause apple engineers didn't feel like making any tools for this, or even leaving a possibility to do this...
Also these 100 UDIDs you are talking about are given FOR A YEAR! SO I will have to carry on these testers all the time all the same without possibility to change them cause I don't have ahough UDIDs?
I don't wonder any more why Blizzard does not develop for iphone...
Steve Jobs, do you hear mortals? Helloo? C'mon, dude...
Do you have an app on the store already? How many testers have you used?
You may want to look in the enterprise program if you want ad hoc distribution to more than 100 devices, it may better fit your needs.
Do you have an app on the store already? How many testers have you used?.
Sorry, i forgot what you said in first post.
So you want to go from 0 testers for your first app to 500 for the second one?
Everybody around here with many apps in the store seems in agreement that 100 is plenty enough, especially if you don't have the structure to manager that many testers. What kind of quality test reports are you going to be able to analyze from that many testers if you're alone(are you?) working on this?
Do you have an app on the store already? How many testers have you used?
You may want to look in the enterprise program if you want ad hoc distribution to more than 100 devices, it may better fit your needs.
Can I have a link?
I had more than 20.000 testers. Yet I assume everyone who bought an app that crashed were testers. And now I am investing a hell ton of money to make a big content update to apologise for this.
I just need all creash reports that will be generated by their iphones... Cause 1 creash report may not help, but 10, 20, 30+ will definitely help...
Quote:
Originally Posted by jptexas
Sorry, i forgot what you said in first post.
So you want to go from 0 testers for your first app to 500 for the second one?
Everybody around here with many apps in the store seems in agreement that 100 is plenty enough, especially if you don't have the structure to manager that many testers. What kind of quality test reports are you going to be able to analyze from that many testers if you're alone(are you?) working on this?
I had more than 20.000 testers. Yet I assume everyone who bought an app that crashed were testers. And now I am investing a hell ton of money to make a big content update to apologise for this.
I dont know the details of the program, but it is meant for in house development. So you would also need to keep your regular developer program for app store release.
I could be wrong, but don't think this is possible.
I have done it to many of my clients who has a specific release date. I tested them using a promo code before release date and even once had to update before going live. So I can guarantee that this works
Dude, I have a problem. Sometimes my app behaves strange and instead of adding something to player it takes something. But this happends extremely rare and all my team could not spot an issue.
But I have complains coming in every day about this issue.
What should I do? Use all UDIDs given for a year to some unknown testers that can even not do the work or get boring in a day, or hire 50 full-time testers?
But it's now absolutely clear why iTunes is working THAT ugly...
And we have to pray about 10-days approval time...
And there is a discovery issue not solved within 2 years...
Eric Schmidt sems to have a nice chance?
Quote:
Originally Posted by vigu360
I have done it to many of my clients who has a
specific release date. I tested them using a promo code before release date and even once had to update before going live. So I can guarantee that this works
Dude, I have a problem. Sometimes my app behaves strange and instead of adding something to player it takes something. But this happends extremely rare and all my team could not spot an issue.
But I have complains coming in every day about this issue.
What should I do? Use all UDIDs given for a year to some unknown testers that can even not do the work or get boring in a day, or hire 50 full-time testers?
But it's now absolutely clear why iTunes is working THAT ugly...
And we have to pray about 10-days approval time...
And there is a discovery issue not solved within 2 years...
Eric Schmidt sems to have a nice chance?
Honestly why don't you try the enterprise program. This is specifically for some one who needs lot of testers. I can assure you 10 good exp beta testers(not some kid next door) can nail down a bug or crash provided you provide them with necessary devices and knowledge of different software versions.
Honestly why don't you try the enterprise program. This is specifically for some one who needs lot of testers. I can assure you 10 good exp beta testers(not some kid next door) can nail down a bug or crash provided you provide them with necessary devices and knowledge of different software versions.
Ok guys, I just needed to make sure that there is no proper way to do this and that I am not a complete dumb that overlook obvious... I will use some workarunds but just wanted to be sure.
No beta test or I am completely dumb?
)
Linear Mode
