How to develop / test flight models

[+mue 715]

A question to the FM gurus: How do you develop and test your flight models?
I'm especially interested in how to gather telemetrie data (e.g. velocity, height, acceleration, turn rate, ...) and how to plot graphs from those data (e.g. velocity over time, height over time, ...).
I know about the debug display. But is there a way to output this data into a text file?

[+baffmeister 1,205]

I think there have only been a handful of SF players over the years that could actually build what might be considered a HiFi flight model and even they would be limited somewhat by the games flight engine.  I'm definitely not an FM guru and what I do is mainly just some basic lift/drag calculations while relying on assorted ThirdWire tables to at least supply some normal behavior. Having said that, I do spend a lot of time trying to track down real world information to build and test the FM's.

I don't know of a way to output telemetry from a test flight. If you have a flight manual for the the aircraft, it's relatively easy to test the plane in game to see if it's close to some key numbers. Knowing the proper climb profile from the flight manual, you can check climb time to altitude. With hud in debug, you can check available "G" at various speeds. If you have a sustained "G" chart, you can check that to see if the FM is close to the real plane. Some of the older flight manuals are surprisingly light on information but even checking the take off distance and speed at a specific weight can be a good cross check for an FM.

It sounds like you might be interested in a computer program based approach to flight models. I have heard of some people using excel type programs for calculating some coefficients but I'm no help there. There are some software gadgets that can make some basic lift/drag calculations. I used one tool called foilsim to try and generate CDL tables for cambered airfoils but the results always seemed a bit iffy. I'm trying a different math based approach now that seems better but still requires certain "assumptions" on my part that may or may not be correct. I used the new  approach on the P-47 and it seemed to work reasonably well. 

I think an actual HiFi FM should demonstrate aircraft specific handling qualities but that's a whole different realm compared to just hitting the important numbers. ThirdWires FM's are probably quite good at that in a general way but seem to lack the real world behavior when at the limits. It is possible to "tinker" with the headline numbers to generate certain behavior but that's a long way from getting type specific behavior over the entire flight regime. Maybe DCS is getting close to that but I think only an actual pilot who has flown the type would know for sure.

Not much help but maybe you could explain your intentions in a bit more detail?  Someone else might know of some programs that would be useful when building FM's.

 

 

[+streakeagle 871]

The best way to assess flight models is to know the math behind the flight engine and build a tool that reads the FM ini file and then generates the performance charts you are interested in evaluating. A long-long time ago, before I had a son, I created such a tool for SFP1. It was primitive and used brute force solutions that would light up your cpu for a while, but it worked. At one point, I needed to incorporate some changes to reflect what TK had added in a patch, and a nasty bug appeared in my code that I could never fix. I wanted to build a new tool from scratch that would support SF2, but I doubt I will ever have the time/motivation to do so. The next best thing is to build a spread sheet that allows you to manually solve the same equations my tool solved. It just takes time to build the initial spreadsheet, then go through the iterations of changing the speed (in mach) and/or other related parameters to find the solution to the problem. The only other solution is to fly in debug mode with a zero burn rate for fuel so that the weight doesn't change while you are flying and observing measured parameters.

[+streakeagle 871]

There are some holes in the flight engine, but overall the SF2 flight engine can produce outstanding results if it is fed accurate data. I was able to get SFP1 to get very close to hitting the numbers for the F-4B over the entire altitude and speed range covered by the charts in the flight manual.

Edited February 18, 2018 by streakeagle

[+mue 715]

Thank you very much for the information!

So it seems, the only way to get the flight data from the game engine is to observe and manually note the desired flight data parameter from the hud debug output while flying the aircraft.

Mmmh...I have an idea: Why not record a video of the flight (e.g. via shadowplay or the like) with enabled hud debug mode. Then afterwards postprocess the video to extract the data from the video images. I think it shouldn't be too difficult to write a small tool in python by using opencv and a ocr library to extract the data. Then we have access to all hud debug parameter over the whole flight time.

My reason for getting the flight data is the following:
I wonder if it's possible to implement the SF2 flight model engine with/in JSBSim (used e.g. by FlightGear) since AFAIK both use aerodynamic coefficient buildup method to compute aerodynamic forces and moments. Thereby the coefficient tables from the SF2 ini files could be used in JSBSim.
And if I decide to give it a try (implementing the sf2 model in jsbsim) I wanted to know how to test/compare both models. Getting the needed flight data from JSBSim is easy but to get the flight data from SF2 I didn't know if there was another way besides the hud debug output.

But why implementing the SF2 FM engine in JSBSim?
1.) To teach myself something about aerodynamics, aircraft control and simulation,...
2.) The JSBSim model can then be used to assess and develop flight models for the SF2 engine. Similar to the tool streakeagle wrote.
3.) If someone is interested in building a (maybe community driven, open source) combat flight simulator, he could use jsbsim/flightgear and use already available coefficient tables from SF2 inis.

[KJakker 901]

Mue, it would be interesting if a tool could be created that could read the all of the aircraft performance data in the aircraft data files and generate Energy Maneuverability diagrams for them based upon that information. Then have it where you could select more than one aircraft and it would overlay the charts of selected aircraft's performance for easy comparison and contrast. That would allow a player or modderto  compare how various aircraft in his install relate to each other as well as make it easier to create flight models.     

[+streakeagle 871]

I looked at JSBSim when I was evaluating Flight Gear flight models. I think you will find that the numbers will not directly translate between the two flight engines.

TK uses extensive tables so that you can plot correct, empirical data. So, if you have a NASA wind tunnel study or better yet a collection of curves/data points from live flight testing, you can usually tweak TK's flight models to match those results.

If I recall correctly, JSBSim was much simpler. You start with a core set of numbers and the flight engine estimates all of the other values at various machs/altitudes. I don't recall how JSBSim handles transonic/supersonic mach numbers, but I don't recall being impressed.

TK's flight models suffer first and foremost a problem with resolution. The original SFP1 flight models were tabulated in increments of 0.4 Mach. In general, this was adequate from Mach 0.4 to 0.8 and Mach 1.2+ where the curves are fairly linear and flat. If you fed the tables the correct numbers, you could get magical "1%" accurate flight models that seem to be the "gold" standard for flight sims. But a lot of things change quickly at very specific Mach numbers over the Mach 0.8 to 1.2 range. To model transonic fighters like the MiG-15, MiG-17, and F-86 accurately, the interval needs to be Mach 0.05. Even Mach 2 fighters like the F-4, MiG-21, and F-105 benefit from high resolution intervals over the transonic range. If TK's flight models could have been so much more accurate if they had supported varying the resolution as needed instead of a fixed interval between all data points. It is a pain in the butt to develop flight models with the original Mach 0.4 interval, and insufferable to use an interval like Mach 0.1 or smaller.

The other problem with TK's flight models is that they can't show interference, such as between the wing and tail. The effectiveness of the tail can be greatly effected at higher AoA by the wing's airflow. The table for the horizontal tail (elevator, stabilator, etc.) is pretty straight forward based on the Mach and/or angle of attack. But the wing can cause the tail to see different speeds and/or angles than the air which originally hit the wing. So you have to choose between tailoring the tail control tables to normal flight conditions such as level flight or to reflect high AoA conditions. The F-100, F-101, F-104, and F-4 are all subject to problems based on this. In my F-4B flight model, I simply ignored the interference effects which combined with the Mach 0.4 table intervals left it quite inaccurate over the transonic range, but still did great at Mach <= 0.8 and ok at Mach >= 1.2.

[+mue 715]

19 hours ago, KJakker said:

Mue, it would be interesting if a tool could be created that could read the all of the aircraft performance data in the aircraft data files and generate Energy Maneuverability diagrams for them based upon that information.     

I don't think there is a simple analytical solution for this.
To create E-M-diagrams you have to fly the aircraft at the altitudes you are interested in over the whole velocity range. For different velocities you then determine the max g you can pull in a level turn. From g and true airspeed you can then determine the turn rate and radius at that velocity. To help gathering this data, this is there my proposed tool for postprocessing the debug hud output can come handy.

[+mue 715]

11 hours ago, streakeagle said:

TK's flight models suffer first and foremost a problem with resolution. The original SFP1 flight models were tabulated in increments of 0.4 Mach. [...] If TK's flight models could have been so much more accurate if they had supported varying the resolution as needed instead of a fixed interval between all data points. It is a pain in the butt to develop flight models with the original Mach 0.4 interval, and insufferable to use an interval like Mach 0.1 or smaller.

I don't see what the problem with a fixed data intervall is, other than that you have to also input high resolution data in the "uninteresting" not much varying Mach areas.
Or is it, that the game engine can't handle high resolution tables in general?

Edited February 19, 2018 by mue
spelling

[+baffmeister 1,205]

4 hours ago, mue said:

I don't see what the problem with a fixed data intervall is, other than that you have to also input high resolution data in the "uninteresting" not much varying Mach areas.
Or is it, that the game engine can't handle high resolution tables in general?

I'm not aware of any limitations regarding the resolution of the data but how much is too much? Here's a stall table from one of the old AvHistory WW2 FM's, circa 2009. I think these were made for SF1 at the final patch level.

StallDragTableNumData=255
StallDragTableDeltaX=1.41732283464567
StallDragTableStartX=-180
// StallDragTableData=1,1.192,1.389,1.597,1.821,2.065,2.336,2.638,2.976,3.357,3.784,4.264,4.789,5.345,5.915,6.484,7.036,7.555,8.026,8.435,8.783,9.073,9.307,9.487,9.616,9.696,9.73,9.72,9.669,9.579,9.452,9.291,9.099,8.878,8.63,8.357,8.064,7.75,7.42,7.076,6.72,6.354,5.982,5.604,5.225,4.846,4.47,4.1,3.737,3.384,3.044,2.72,2.413,2.127,1.864,1.627,1.417,1.238,1.093,0.983,0.912,0.881,0.894,0.953,1.06,1.214,1.413,1.651,1.924,2.229,2.562,2.918,3.293,3.683,4.085,4.494,4.905,5.316,5.722,6.119,6.503,6.869,7.214,7.534,7.825,8.082,8.303,8.488,8.637,8.751,8.829,8.871,8.878,8.85,8.786,8.687,8.554,8.385,8.182,7.944,7.672,7.369,7.037,6.678,6.295,5.89,5.467,5.026,4.571,4.105,3.635,3.167,2.666,2.18,0.869,1.015,0.979,1.007,1.001,0.996,0.998,1,1,1,1,1,1,1,1,1,1,1,0.999,0.999,1.001,1.001,0.995,1.004,0.973,1.244,1.665,2.083,2.498,2.914,3.331,3.75,4.197,4.666,5.148,5.631,6.104,6.557,6.979,7.359,7.687,7.951,8.143,8.263,8.32,8.319,8.268,8.173,8.042,7.881,7.693,7.482,7.249,6.996,6.727,6.442,6.145,5.837,5.522,5.2,4.875,4.549,4.223,3.901,3.584,3.274,2.975,2.688,2.415,2.159,1.922,1.706,1.514,1.348,1.209,1.101,1.025,0.984,0.978,1.006,1.066,1.156,1.276,1.423,1.597,1.795,2.017,2.261,2.524,2.807,3.106,3.422,3.752,4.094,4.448,4.81,5.178,5.549,5.92,6.289,6.654,7.011,7.359,7.694,8.013,8.315,8.597,8.855,9.088,9.292,9.466,9.606,9.71,9.775,9.799,9.778,9.712,9.595,9.428,9.205,8.926,8.587,8.185,7.719,7.198,6.637,6.055,5.467,4.892,4.346,3.847,3.402,3.008,2.659,2.349,2.072,1.824,1.599,1.39,1.192,1
StallXacShiftTableNumData=255
StallXacShiftTableDeltaX=1.41732283464567
StallXacShiftTableStartX=-180
StallXacShiftTableData=-2.539,-2.517,-2.495,-2.474,-2.452,-2.43,-2.408,-2.386,-2.364,-2.342,-2.32,-2.298,-2.276,-2.254,-2.232,-2.21,-2.188,-2.166,-2.144,-2.122,-2.1,-2.078,-2.056,-2.034,-2.012,-1.99,-1.968,-1.946,-1.924,-1.902,-1.88,-1.858,-1.836,-1.815,-1.793,-1.771,-1.749,-1.727,-1.705,-1.683,-1.661,-1.639,-1.617,-1.595,-1.573,-1.551,-1.529,-1.507,-1.485,-1.463,-1.441,-1.419,-1.397,-1.375,-1.353,-1.331,-1.309,-1.287,-1.265,-1.243,-1.221,-1.199,-1.177,-1.155,-1.134,-1.112,-1.09,-1.068,-1.046,-1.024,-1.002,-0.98,-0.958,-0.936,-0.914,-0.892,-0.87,-0.848,-0.826,-0.804,-0.782,-0.76,-0.738,-0.716,-0.694,-0.672,-0.65,-0.628,-0.606,-0.584,-0.562,-0.54,-0.518,-0.496,-0.474,-0.453,-0.431,-0.409,-0.387,-0.365,-0.343,-0.321,-0.299,-0.277,-0.255,-0.233,-0.211,-0.189,-0.167,-0.145,-0.123,-0.101,-0.078,-0.055,0.006,-0.001,0.001,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0.001,-0.013,-0.035,-0.057,-0.079,-0.101,-0.123,-0.145,-0.167,-0.189,-0.211,-0.233,-0.255,-0.277,-0.299,-0.321,-0.343,-0.365,-0.387,-0.409,-0.431,-0.453,-0.475,-0.496,-0.518,-0.54,-0.562,-0.584,-0.606,-0.628,-0.65,-0.672,-0.694,-0.716,-0.738,-0.76,-0.782,-0.804,-0.826,-0.848,-0.87,-0.892,-0.914,-0.936,-0.958,-0.98,-1.002,-1.024,-1.046,-1.068,-1.09,-1.112,-1.134,-1.155,-1.177,-1.199,-1.221,-1.243,-1.265,-1.287,-1.309,-1.331,-1.353,-1.375,-1.397,-1.419,-1.441,-1.463,-1.485,-1.507,-1.529,-1.551,-1.573,-1.595,-1.617,-1.639,-1.661,-1.683,-1.705,-1.727,-1.749,-1.771,-1.793,-1.815,-1.836,-1.858,-1.88,-1.902,-1.924,-1.946,-1.968,-1.99,-2.012,-2.034,-2.056,-2.078,-2.1,-2.122,-2.144,-2.166,-2.188,-2.21,-2.232,-2.254,-2.276,-2.298,-2.32,-2.342,-2.364,-2.386,-2.408,-2.43,-2.452,-2.474,-2.495,-2.517,-2.539

I have to wonder what affect a large engagement of WW2 planes with data inis like this would have on game play.

[+mue 715]

1 hour ago, baffmeister said:

I have to wonder what affect a large engagement of WW2 planes with data inis like this would have on game play.

The time complexity of the lookup in a table with a fixed data interval is constant. That means the number of entries in the table shouldn't matter.

[+streakeagle 871]

The problem with a small interval is correctly filling out the data without errors. It is tedious to fair in all the points compared to the 0.4 Mach interval. But the gains in accuracy in areas of steep transitions is critical. All aircraft have a critical Mach number where the wave drag spike's up. When you are talking transonic/marginally supersonic aircraft, accurately modeling wave drag is critical to accurate performance above Mach 0.75.  For aircraft like the MiG-15 and F-86, it will define their top speed. The F-86 has a higher number and is therefore more controllable at speeds the MiG-15 can barely attain and able to attain speeds the MiG-15 cannot. The problems caused by this region are the reason for coke-bottle (area-ruled) fuselages. The specific excess power needed to go supersonic doesn't exist if the aircraft isn't designed to minimize transonic drag and provide decent control/stabilization in that speed range.

As for building EM diagrams, my old tool, AIDE, would plot height-mach sustained and instantaneous g curves and easily could have been modified to show specific excess power curves. You just had to weight while it searched for the data minimum and maximum data points over the Mach range for a given altitude and over the complete altitude range. The math isn't really hard, just time a consuming re-iterative process. The first problem is to build a data structure that can be loaded with every possible value in the data ini files, then you have to read in the data ini file and parse it to populate the data structure, then you choose the table/parameters you seek, then the program performs the calculations to balance thrust, drag, lift equations based on the provided parameters, then a plot of the data is displayed.

The problem starts with the number of variations that have to be handled when parsing the ini file into a useful data structure. The re-iterative calculations that produced useful performance charts could cause a memory leak, an infinite loop, or just outright CTD or BSOD, so it was important to carefully test and error-trap each subroutine that performed any data object creation/destruction and/or complex calculations. I used to have the time an patience to develop such a tool. Now, I don't want to perform any work, I just want to enjoy the end result and better people than me are providing great flight models for me to enjoy in both DCS World and FSX/P3d. I am far less critical/demanding than I once was because of how well I understand the limitations involved in trying to produce a truly accurate flight model on a PC. I don't expect anything close to perfection, not even a true "1%" error margin. But I have zero tolerance for oversimplification such as the original LOMAC/Flaming Cliffs flight models that made Strike Fighters look like a real NASA engineering program.