Beta-testing

[MrFrench]

First of all, let me make it clear that in this thread my intention is not to criticize the beta-testers or LO:MAC. I like this sim alot and I think that a program this complex for this price is totally unbelievable in today's society.

 

What I was wondering is this:What do the beta testers do exactly?

For example, how many are there for a program like LO:MAC? What kind of tests do they perform, do they simply play the game or tests specific things?

 

What prompted this question is the whole problematic (discussed at lenghts in many threads, mainly PAPADOC's thread on the AMRAAM) concerning the PK of ALL the missiles with self guidance systems, wether IR or internal radar. Is it normal that this was not discovered during beta testing?

 

Anyways, let me remind you that I am just looking for information on the subject and not trying to criticize anyone. I would just like some feedback from someone who is familiar with the process of beta testing.

 

Ian

 

Note: message also posted in UBI general forum, so don't be surprised if you are a user of both forums.

[switch]

I saw your post on UBI`s forum. It`s sliding fast into oblivion of page 2,3, 4 ...

I`m sure many people asked (themselves) that question many times.

...I don`t know. I know what they should be doing. And if they did have some

input, who knows how the devs responded?

[MrFrench]

I know, it's difficult to keep a post up there, especially with many persons using the general forum as the tech support...

 

Yes, I am curious on the relationship between the testers and the developpers.

 

Ian

[+Fates]

Simply put, Beta-Testing is making sure that the product performs in a manner that meets the developers description and design

 

In that statement, things like making sure all the mechanics of the game work to the developers design. This may include confirming that everthing works properly, is graphically correct, and performs their intended function per the developers design. This can be blanketed on to include aircraft, vehicles, missles, guns, flight instruments, etc..etc.. or just anything that does something.

 

The game is tested at the Beta stage by outside players that are generally not part of the design/development team. They are a mixed bunch that represent the public in the form of computer equipment, connection speed, and they generally have a certain amount of gaming experience to know what to look for and raise questions about. I found many times where I thought I found a bug, only to discover that it was designed that way because.....insert reason.

 

As bugs are found, they are reported to the developers and there is generally a QA session where that bug will be disscussed and confirmed by other testers. The developers, knowing the flow chart of code will also know what questions to ask the tester so a resolution can be found. Troubleshooting any problem starts with a good understanding of the code, and somethings are simple to change, others are much more complex.

 

Generally, developers will tackle several bugs and repair them and then release a new build. This new build is like having a whole new game, and just because something worked in the 1st build don't mean it still works with subsequent builds. This can make a Beta-testers job quite tedious becuase he/she must still go back and make sure everything...EVERYTHING...still meets the developer design. With any luck, the new code hasn't broken other things and the new build will be better then the last. However, somethimes changing code at the developer stage can break more things then were fixed.

 

This cycle repeats until:

 

Eventually, Producers enter the picture, and say "WHEN"? The Developer says "Soon?"...and the Beta-Testers say "What?"...The Producer then says "NOW!"...the Developer says "Please...can I have more time?"...the Developer's say "NO!"...and the Beta-testers say "I'll be here for the patch"

 

To answer some of your specific questions:

For example, how many are there for a program like LO:MAC? What kind of tests do they perform, do they simply play the game or tests specific things

I can't say specific, but I would guess at 20+ Beta-testers plus the development team. They perform as mentioned above. They don't just play the game, although they may have some nights were they have fun, most of it is truely work. For example you my find a bug that only happens at a certain point in the mission. Well, you might not be able to zoom to that point to test the bug...and you'll spend 30 minutes attempting to duplicate it. Plus 10 other testers will do the same thing. Not just onace...but several times. They will try to trouble shoot themselves (which is good) and they may spend an 8 hour day just trying to theorize what the problem could be. They play the game and LOOK for bugs. Once those are found...they are reported and then other testers will attempt to duplicate and bring the bug to the Devs attention.

 

concerning the PK of ALL the missiles with self guidance systems, wether IR or internal radar. Is it normal that this was not discovered during beta testing?

 

My guess is that it was brought up in testing, but somewhere there has to be a line drawn and the game must reach the consumer...with or without bugs. This is something all of us hate as consumers, but it's a fact of life that I don't think is gonna go away anytime soon. Any problem with a game has to be expained to the Producer which have deadlines to meet to get the product to the storeshelves. Sometimes, like in the case LOMAC, I believe the developers were having a difficult time with certain modules of the game...they begged producers for more time and LOMAC was delayed to allow a better game to reach the consumers. Had they released it early with larger more complex bugs, the product would have been met with larger dissapointment by the consumer and decreased sales.

 

 

Cheers!

 

Fates

[Seawolf]

Software is going to have bugs and testers can only test so much. Whats important is once the product is released the Dev team is allowed to continue working on it.

[MrFrench]

Great answer Fates!

 

Thanks for all the good info, that's why I like this board, people in the know take the time to write informative answers!

 

I especially like the:

 

Eventually, Producers enter the picture, and say "WHEN"? The Developer says "Soon?"...and the Beta-Testers say "What?"...The Producer then says "NOW!"...the Developer says "Please...can I have more time?"...the Developer's say "NO!"...and the Beta-testers say "I'll be here for the patch"

 

LOL!

 

Ian