FPS Success in SoH

An FPS test with 103 heroes and 103 Nagdellians under the Arch.  FPS is 31.6.  Blue Man is our lead coder’s favorite test character.

Hello, Ship of Heroes supporters!  This newsletter is quite different from what we’ve published in the past.  First off, this is part one of a series of four newsletters in which we’ll share some really good news.  Second, we’re going to do something that is very rare for a game at our stage – we’re going to publish some of our current, internal FPS numbers, and we’re going to share what we think these numbers mean.

Over the last couple of years, we’ve developed three proprietary techniques for making Ship of Heroes faster, measured in frames per second or FPS.  High FPS is difficult to achieve, particularly in MMOs, because of the large numbers of players that should be on the screen at once.  Think of a mothership raid in COH or fighting a worldboss in several other MMOs, with fifty or more players in close proximity, fighting enemies.  An MMO needs to keep reasonable FPS in this situation in order to deliver that intense player experience.

Our first technique was rebuilding our character models to shrink the file size, number of textures, and other parameters.  Characters look the same or better, so this is good news all around.  Since we’ve completed this rebuild, we now have “new,” faster characters and “old,” slower characters that essentially look the same.  When the game launches, we will only have “new” characters.

The second thing we’ve built is a 4th generation of our CROM system for map optimization.  We now have very advanced methods for making our city use less client-side graphics capacity for rendering the environment.  The third system is runtime mesh-merging.  We are holding this one in reserve, since it’s not an improvement until there are 75+ players on screen, but we may use it in raids and invasions.

Elevated view of the testing scene.  Each side gets one non-damaging power to test FX.

Ship of Heroes was always designed to be highly optimized, with the goal of delivering a good FPS range for our players.  Even with a pretty efficient starting point, an MMO still needs these advanced techniques to deliver a solid player experience at the end.    So this kind of technical progress report ought to be a major part of the development of any MMO.  For those who are keeping track, it has been about a year since our last FPS report, when we published our Night of the Clones video.

Let’s take one example.  Consider the impact of the new tools we’re employing on a situation with 50 heroes in close proximity at 1920×1080 resolution.  Our target is generally to deliver 30 FPS or greater, as in most successful MMOs.  These FPS numbers are from our lead coder’s desktop computer, with a 1060 NVidia graphics card, and they illustrate why CROM 4 and our latest character optimization are important.

FPS For 50 Heroes in Close Proximity (NVidia 1060)
Old Character Models New Character Models
No-CROM 4 20 22
CROM 4 51 64

64 FPS for 50 heroes on the screen is a great result.

These results, like all of the data in this newsletter, are preliminary.  But we always use a packaged game in a client/server situation to simulate what a player experiences – we’ve determined that FPS testing in the editor is unreliable and wildly optimistic for technical reasons.  We chose one of the slowest spots on the map of Apotheosis City.  And we have these heroes in a box directly in front of the dev’s character, so we’re mimicking a raid situation with nearby characters; distant characters barely affect FPS by comparison.

So what does this mean?

  1. Even though we had efficient environmental and character designs to begin with, we needed these upgrades to match the FPS range typical of the successful MMOs out there today. And our techniques work!
  2. We needed to take the time to develop and perfect CROM 4, because without CROM the character optimizations are insufficient to get us to 30 FPS in a high density player scenario. This is still ongoing, and it has pushed back our schedule a bit, but we’ll issue a new schedule next month, which will include a full beta launch of SoH at the end of 2019.
  3. After we developed CROM, we needed to optimize the character models – this was really our fourth round of that effort too – in order to deliver another important increment of FPS improvement.
  4. What this shows is that, like most MMOs, we cannot simply buy art or code assets created for non-massively-multiplayer games and drop them into ours. Everything needs to be carefully vetted, and usually rebuilt, to be useable, or the FPS will be too low to give a good player experience.  And no feature or content in the world will ever make up for the loss of immersion from lag and low FPS constantly distracting you from a game.

An up-close scene of nighttime testing.

We’re listing some additional preliminary data below to expand the context of the successes we’ve had with optimizing Ship of Heroes.

All of the numbers reported below use CROM 4 and our new models, unless otherwise noted, and all but the last use 1920×1080 resolution. Because we conducted many, many tests over the course of nearly two years during the development of our CROM process, we know that CROM 4 is our best system.

This table illustrates how FPS varies with the number of unique heroes on the screen being rendered simultaneously, in close proximity.

Varying FPS with Number of Heroes on Screen in Close Proximity (NVidia 1060)
Number of Heroes 1 10 20 30 40 50 60 70 80 90 100 110 120 130
FPS 80 77 73 70 67 64 61 58 55 51 47 42 39 35

Note: We have some runs in which these numbers are a bit higher.  No FX or combat is occurring in this example.

Next we look at situations in which large numbers of unique heroes are mixed with large numbers of Nagdellians, one of several species of enemy raiders in SoH.  This time we have minimal FX and powers being used, and intermittent movement of all heroes and enemies.  There are four unique Nagdellian models.  All heroes are highly customized, and were created in our character creation tool (CCT) from three base models.  The heroes support all the morphs, clothing, and data that normal characters support; they are not NPCs.

Varying FPS in Simulated Invasion Scenario (NVidia 1060)
Number of Heroes 15 30 45 60 75 90 105 120
Number of Nagdellians 15 30 45 60 75 90 105 120
FPS 61 53 48 42 37 31 25 21

Note: we are currently planning 50-man invasions with 50-70 enemies as a typical maximum.  If these FPS numbers hold up, we may be able to go even higher.

We often use torches as an example power for heroes.

Finally, let’s look at some results for an older computer, this time using two of the other tools we’ve coded for Ship of Heroes, which are resolution settings and a significance manager (SM).  In this case, resolution has been reduced to 1280×720 and the SM has been set to try to deliver at least 45 FPS, even though the computer would not deliver this result normally.  If activated, SM runs in the background and makes automatic changes to shadows and other factors to try to give a player a desired FPS result.  There will be more on this feature in a future newsletter. The high resolution without SM results are also provided for comparison.

The center of the city under the Arch is busy!

Varying FPS with Number of Heroes on Screen in Close Proximity (ATI Rx 480)
Number of Heroes 1 10 20 30 40 50 60 70 80 90 100 110 120
FPS at 1920×1080 59 43 48 43 38 32 28 24 22 20 18 16 14
FPS at 1280×720 with Significance Manager ON  

105

 

92

 

75

 

57

 

51

 

46

 

42

 

37

 

32

 

25

 

23

 

20

 

18

Note: these numbers show that the SoH dev team is building tools and options for players to adjust the FPS of the game with visual trade-offs.  In this example, shadows are dropped at around 50 players to raise FPS.

75 civilians are being rendered in this shot with relatively modest impact on FPS

Technical Notes:

  • We spawn heroes by grabbing a random character from a list we’ve created using our CCT. Old models are picked from a list of 20 different heroes, while new models are chosen from one of 11 different heroes. The characters are in essence identical to a player character, using the same code, but they are controlled by an AI.
  • The number of civilians and cars in any scene has a relatively small impact on FPS, compared to players.
  • The Ship of Heroes day/night cycle has very little FPS impact. This outcome required months of work by skilled programmers.
  • Movement by hero characters has a very modest cost in FPS.
  • FPS measurements made in the editor are meaningless. By choosing editor parameters, we could make the numbers into anything we wanted them to be.  Also, part of the FPS depends on latency associated with off-premise servers, and this should be included to reflect the player experience; it is, in our numbers.
  • FPS numbers vary significantly depending upon how much of Apotheosis City is visible, and how much is hidden in the background.
  • FPS numbers vary enormously, even when our test characters are stationary. They fluctuate slightly, typically by plus or minus 2-4 FPS.  In order to obtain comparable numbers, we try to perform our tests exactly the same way each time, facing in the same direction.
  • For tests with powers and FX, we used a simple power, Torch, for the heroes and granted one power that does no damage to the Nagdellians so they won’t reduce the number of characters on-screen by killing them.
  • Prior to the development of CROM 4 we began seeing a disparity in our frame rates, for different graphics cards. The ATI cards were significantly outperforming Nvidia cards. This was resolved with our fourth revision to the CROM system which resulted in major performance boosts for both cards and brought them back in line with what we expected.
  • Time spent developing CROM 4 and optimized models is worthwhile; this foundational tech cannot be easily added to the game at a later date, and it is essential to the player experience for our kind of MMORPG.

Based on these tests, we think it is reasonable to assume that we can conduct a login test with 50-100 unique heroes, and that we can conduct a raid test with at least 50 heroes and 70 Nagdellians, with the expectation of success.