As part of my studies on Game Engineering in Saxion University, I did a 6-months research
project
(January 2020 – June 2020) called “Minor Skilled”.
The final product a game created in a 3D Game Engine, implemented from scratch in C++ using
ECS as
the main architecture design.
The purpose of this report is to share my research by answering the following question: What benefits does ECS architecture bring to Game and Engine development?
I will describe the work showcased in this report in three main parts. The first part comprises of a brief introduction to the Enity-Component-System architecture and show my implementation of this design, which I named Feather. In the second part, I explain the 3D engine called Crow and how feather was used to implement these complex engine systems. Last part of the work showcased will be about the game called Graveyard Warz, and some of the gameplay system built using ECS architecture.
The main goal of this project was to research on Entity-Component-System architecture and answer
the
question: What benefits does ECS architecture bring to Game and Engine development?
To answer the main question I had to research and answer the following questions:
Traditionally, games were used to be programmed in a object oriented desing (OOP) or inheritance
design. For example, a class Creature inherits the base Gameobject class and then there
might be a class
Human and Orc which inherit Creature class and so on. Then every frame the
program goes through all the Entities and call some
methods
like Render or Update.
The first problem with this approach is that our architecture is
rigid, meaning that if another class Necromancer wants to be both an Orc and
a Vampire, the whole program architecture has to be redesigned.
The second problem with the OOP is related to the performance. When iterating through all these Entities
calling
Update, the CPU is jumping in memory and pulling data in cache,but since our entities are
scattered
around in memory that cache is not used the next frame thus the
CPU has to go and fetch that data from RAM every single time, so essentially our CPU power is wasted
due to the fact that you have to wait for the data to process.
Nowadays the industry has shifted to Component System design for solving the problem
mentioned above.
For example, in Unity's Component System [1]
there is a Gameobject to which components can be
added as building blocks and then Update is called on those
components every frame.The first issue is solved since, Vampire and Orc components
can be attached to a GameObject which will then act accordingly. However the
second problem still remains, since the program is still jumping in memory
and calling Update on all of these components.
If you have a look at the picture in the right, which represents how a component system memory layout
might look like,
the problem becomes more noticeable. The program is moving all around memory in one frame and wasting
CPU cycles.
In order for the second problem to be solved, the memory layout needs to be re-organised so thatthe
components
that are iterated regularly are tightly packed together.
For example if all gameobjects are needed to move, all the transforms would be iterated and the
position would be changed.
However these transforms have to be grouped in memory together.
Then when the CPU request one transform it will fetch a bunch of them (as much as it can fit
in
one cache line) and store them in cache for the next frame.Then next iteration
another transform is needed,they might are already stored in cache hence saving the CPU a lot of
time.
Introducing Enity-Component-System approach, in this design,an entity is just an
index to
a group or collection of components. All the components are stored in tightly
packed array and those contain only data and they are also known as POD(Plain Old
Data)
structures. Instead of components updating themselves we have systems
which iterate through all entities which have a specified set of components and update their data.
For example the Rigidbody system will operate only on entities which have the Rigidbody and
Transform component.
If an entity has neither or just one of these components the system wont update their data.
Unity Engine is now also moving towards this architecture with their new DOTS [4] api.
In this section my own implementation of ECS architecture called Feather will be explained.
First the overall Architecture will be shown and then speed tests as well as actual code
snippets
will be displayed,
showcasing how it is like developing using Feather.
The full open sourced codebase for Feather can be found here.
In Feather Entities are just an unsigned integers which are used to get the index in a set of components. Components in Feather are just structs of data. Systems is where the functionality is implemented, the user can create their own systems and can iterate through the entities in the world. There are multiple ways of iteration which are shown later in this report.
Except the main concepts like Entities,Components and Systems there are a few more "Classes" that the user needs to understand in order to use Feather:
As mentioned before a key advantage of ECS is providing good memory layout.
All memory that feather uses is allocated in the beginning of the application.There is no
runtime
heap allocations done by Feather. The memory is partitioned correctly and marked as valid or
invalid based on users input. When a component is marked for destruction, the component is moved
and
it wont be updated, however it wont be deleted from memory until the user explicitly needs the
data
to be deleted
(Usually done in the end of the application's lifetime).
Components in feather are stored in a custom collection called ComponentSparseSet.
In Feather a ComponentSparseSet is just 3 C style arrays:
The reason why the 3 arrays are needed is to answer the problem What happens if a component is marked to be removed? The pictures below is needed to show why this is a problem. The picture represents a simplified visualisation of how the components are stored in memory. As said before the ComponentSparseSet contains 3 arrays and those arrays are displayed in a table format. Now the max size is set to 5 elements and it is empty.
So what happens when "C1" is added to Entity "0"(entities being just unsigned integers) to our set?
It is immediately added in the first available slot in the array, in this case it is slot 0.
In the entity array Entity "0" is added to the first available slot too.
Last in the index array, the index at which the entity's component is stored is added
and we store it at the 0 index since our Entity = 0.
Now we have mapped Entity->Index and Index->Entity.This is important when we destroy a component.
Components "C2,C3,C4" are added to the Entity "1,2,3" respectively and the arrays are updated
as shown before.
But what happens if we remove a component from this set?
What happens if "C3" needs to be removed from the set? At first "C3" and "C4" are swapped in the
dense
component array then
the Entity->Index and Index->Entity need to be swapped as well, since next time we need to get "C4"
the
Entity needs to point to the updated index.
In the Entity dense array the same thing needs to be done, so the entity to remove is swapped with
the last entity.
In the index sparse array the entity index needs to be updated.
As it is shown the index sparse array has holes in it but that is not a problem since it is never
iterated
, it is only needed when a component is removed and the set needs to remain packed.
In case the user needs to get "C4", first it's location will be found by using it's Entity value(in this
case 3)
to find the right index.
Then in the index array, the 3rd element is retrieved which is 2 meaning that the "C4" is
located in the 2nd index of the dense array.
Finally "C1" is removed from the set.
Again "C1" and "C4" are swapped with each other and "C4"'s index is updated to point in the location
where "C1" used
to
be.
Afterwards the array will always remain packed, and it can be iterated safely.
The swapping cost
is not that high since its just two C-style array look-ups and the lookup is done only when a
component
needs to be removed which is not that frequent.
However the benefits are big since the data that is active and needed is always contiguous in
memory.
First Feather Vs Object-Oriented approach was tested. Two programs were created which
involved
two different
types of entities, Orcs and Nobles.
In the OOP they are classes inheriting from other base classes. While in
ECS/Feather
they are just two simple components. The behaviour itself is very trivial,
in this case the iteration speed is the focus of the test.
Three different cases were tested, one with 1000 "Orcs" and "Nobles", one with 10,000 and
one
with
100,000. The program is executed for 1000 frames and in order to get a consistent result the
app
is executed 100 times.
The results are displayed graphically in the right. The X-axis shows the number of time
the
program was executed and
the Y-axis shows the time in milliseconds it took for 1000 frames.
In all three cases Feather performs much better then OOP when we iterate
through components/behaviours.
There are two main reasons why OOP performs worst than Feather.
First one is is related to the fact that the data in Feather is contiguous in memory,
thus the CPU cache saves cycles by storing the data we might need the next frame.
The second reason is concept known in OOP as Double Dispatching which in this
case is
a call to the virtual "Update" function. Double-Dispatching
requires a V-table lookup and that is also located somewhere scattered in memory thus
waisting CPU cycles.
The second test performed was against Unity and it's current component system. For
this
test the
steering and flocking behaviour systems build in Unity are the same one as the ones used in
final game in C++.
The unity program only
spawns a bunch of spheres and gives them a target to seek while they try to keep
distance
from each other using basic
O(n2) flocking.
As shown in the video on the right, after 300-400 spheres the frames drop under
30
FPS and at 500 spheres the framerate is very low. The current spheres have
no collider attached to them and the most expensive calculation is done by the Flocking
Behaviour component.
In the second video the same systems are tested in Feather and in an custom C++ Game Engine (Crow). The performance difference is noticeable, now there are way more units, 3-4 times more then Unity while keeping a steady high framerate. After 1200 the FPS drops exponentially since the flocking is done O(n2). The Feather performs better while also having other gameplay systems running in addition to flocking system.
There are many things that could be done to even get more units on the screen. For example the flocking and collision instead of being performed in O(n2) they could be performed in logarithmic complexity by introducing spacial partitioning algorithms. There are also rendering optimisation that could be implemented such as batching for all the meshes. However the goal was to test the same systems running in an equal environment.
In this section, some examples of how to set up and use Feather will be shown.Afterwards how
to
create Entities,Components,Systems
and how to iterate through the components and create functionality will be displayed.
First Feather needs to be set up. In order for Feather to function we need to create
the
3 main registries, the Entity,Component and system registry.
After creating these registries, the next step will be about creating a world which will act as an context and it will
communicate with all the registries. The world needs to be initialized by calling the "Init" function
and passing the registries,
this
way the world is ready to be used and knows which memory it has to operate on.
Since components are just data, they are very simple to create. The only thing needed to create a
component is a struct with our values inside, that's it.
As displayed below, a position component which holds an "X","Y" values and a gravity component
which
defines a "gravityValue" are created.
Systems are not that complicated to create either. As stated before systems define the behaviour of
our
program. The first picture below
displays how to set a the signature of a system. This is only required if systems need to
know
about which entities to iterate on
beforehand and let the World automatically update the entities set for the user.The line below is
telling Feather that the GravitySystem
needs to iterate on entities which have both a Position and a Gravity
component.
This is one of the ways to iterate components in ECS
architecture. In the further sections, other ways to iterate components and what are the benefits of
each
way will be explained.
The second picture below is an example of
how a gravity system would look like. First custom system needs to inherit base System class,
this way we let feather know that this class
is a system and it gives a bunch of functions to override such as Init,Update,Render etc. In the
"Update" function
all the entities which fit this system's signature are iterated and their Position and
Gravity
components are retrieved. Then the Position component is updated based on "gravityValue" of the
Gravity
component.
Another way of iterating components is to use the EntitiesWith function of the World
class. EntitiesWith
is a Query function which returns a list of Entities which contain a given case of components in
this
case a Position and Gravity
component. Internally this function takes the smallest set of the given component and checks which
of
it's entities is located in any of the other
component sets and return those to the user.
After list of entities is retrieved, the entities can be iterated and updated just as in the example
above.
An even easier way to iterate component is using the ForEach function. It takes a function as
a
parameter and provides the references
to the entity and components automatically. This is similar to how Unity iterates components in
their
DOTS api.
Now the components don't have to be retrieved from the entity since they are automatically filled in by
the
world and instead the user can focus on writing their behaviour
inside the lambda function.
The last way to query entities and components in Feather is using the FindEntities function.
This function
will return all the entities that fit
a conditional function given as parameter. As presented in the example below, all the entities
which have a Position component and have
their position's "X","Y" values equal to 0 are retrieved.
Then these entities can be iterated and updated accordingly, in the example below they are just
destroyed.
In order for systems and components to work, the user has to allocate memory for them.
This way all the memory is allocated upfront
so there is no heap allocation at runtime.
If for example a new world needs to be loaded,the same memory can be used,instead of deleting memory and
allocating it again.
Another good thing about this approach is that it minimises null errors in runtime since all of the
memory is valid and partitioned correctly.
Lastly the only thing left to do in order for the application to work is create the entities and
attach
components to them.An EntityHandle
can be created by the world and then any type of component can be attached to it. This components have
to
be allocated before being added otherwise an
error will be thrown.
The last line just updates all the systems registered to the world. And that is everything needed to
set
up and work with Feather.
As mentioned above in this report,the goal is to research the question What benefits does ECS
architecture bring to Game and Engine development?
In this section the focus will be What features does the Crow Engine offer and how were they
build
using Feather(ECS)?
Crow engine provides the basics feature a game engine needs to get the user up and running. It has a
Renderer and Material workflow, window and input API,
Game Loop, ResourceManager for loading and maintaining assets, and support for using Unity as an
editor
tool, however only in this section only the
Renderer and Unity as editor will be explored since this is where the ECS architecture
was
needed to be used the most.
Since one of the core systems of every game engine is it's renderer, naturally the question of how
your
architecture will support your renderer rises.
In this case How to build an ECS renderer and what are it's benefits?
One of the main problems with an ECS renderer is fitting the drawable objects into cache. Meshes
and
materials can be big in terms of byte size and might not fit inside
a cache line so in order to benefit from data-oriented architecture, a way to fit as
much as possible in a cache line needs to be found.
How the Crow Engine approaches the problem is by splitting the data. So in the picture below there
is a
Meshinfo component set and instead of holding
a reference to both a material and the mesh, it holds a pointer to a material array and a mesh
array.
This way the MeshInfo component is small
in size (it only has 2 pointers) but also it points to packed arrays of meshes and materials. The
same
thing is done for materials and shaders, since
multiple materials can use the same shader, a pointer to the shader is stored in the material.
By doing rendering in ECS style the scene can be sorted easily. The Crow Engine supports instanced
rendering and how that works is when a shader is buffered to the GPU,
a list of all the transformation matrices is sent to the GPU for each model and material pair. This way
we save a lot of the draw calls the
GPU would have to do otherwise.
Another benefit of this approach is that if we were doing something to all the materials we would
not
need to touch the mesh part of the memory, instead we just
put all the materials we need in cache.
In order to test the speed of the ECS renderer, the same rendering application was created both
in
Unity and Crow Engine. The application is
10,000 meshes rotating at a random speed. For the test to be fair an unlit material is attached
to
the meshes in both engines so the lighting would
not affect anyhow the performance.
The top video on the right shows the Unity engine result. In unity the app performs around 6-8 FPS
while as it is shown in the bottom right video
the exact same application in the Crow Engine performs in a constant 16-17 FPS.
The main reason why the ECS renderer performs better in this situation is because of how fast
the
cpu is able to iterate and work on contiguous memory. As shown above,
all the Meshes,Materials and Shaders are stored as packed together in memory.
The Crow Engine renderer is not a better renderer then Unity's Renderer, this test is only
showcasing how the data-oriented architecture is more efficient.
There are many improvement that could be done here like Batching and Multi-threaded
Rendering but these were not the scope of the project and did not
fit into the the main goal.
Another reason the custome renderer is able to perform better is not only because of the way
material,shaders and
meshes(in this case) are stored in memory but also because they are sorted
at runtime in order to have as little shader swaps as possible. This way the instance rendering
is
implemented and saves the GPU draw calls but also the GPU does not have
to swap shaders as often, which is a fairly expensive opertation.
Below some of the shaders and materials build with the Crow Renderer are showcased. These materials and shaders are also used in the final Game.
Bling phon lighting model, supporting Directional,Point and Spot lights.
Custom shader and material that fake translucency.
The default texture material of the renderer. Supports Diffuse,Specular and Emission maps.
Crow Engine uses Unity as an editor. All the Crow Engine main components are supported and
serialised from Unity. The Unity scene is serialised into
an XML file format which can then loaded and parsed in the crow engine. The flipping of
coordinate system from Left-Handed to Right-Handed
is done automatically by the parser.
The scene hierarchy is also mantained meaning child-parent
relations are
parsed into the Crow engine as they are in the Unity engine.
The bottom picture in the left shows the scene in Unity and that in the right shows the same scene parsed by Crow engine. The parser will create all the entities and attach all the supported components based on the XML File generated from Unity.
As mentioned before in ECS architecture the components are just strucutres of data.
This makes serialisation of the
components trivial. All the supported engine components work dynamically with Unity's
components.For example in the video on the right
the light component when changed is also chaning unity's light component automatically. This
way changes are visible immediately
in the Unity engine and then can be exported for the Crow engine.
The engine supported components are :
Another feature of the parser is that it can serialise assets from unity. All the user has
to do is place
the texture or model into the right folder in unity and parse the file. The file parset is a
.asset file, which
is a custom serialisation format created for the loading of assets in the Crow Engine. As
shown in the picture on the right,
the format is very easy to read. The "#" is a type identifier meaning anything after
it represents the type of the asset.
Anything after the ":" is the path of that asset.
Once the file is generated from unity, next time the Crow Engine loads it will automatically
load all the assets.
The custom format is very fast to be interpreted as it is written fully in C and since it
has a very simple way of identifying tokens.
Graveyard Warz is a game build using Feather and the Crow Engine. The main reason it was created is to showcase the The benefits of the ECS in Game development. In the Speed Test section of Feather a stress test of the game was showcased, where alot of units were spawned and simulated. In this section, the focus will be how ECS helped in the workflow of developing the game and show how it evolved during development, from the prototype phase to the polishing phase.
Graveyard Warz is a Tug of War game, where you purchase units and lead them to the
enemy's portal.
The player battles against the AI and whoever manages to empty the other's health bar wins the game.
The game is also in 3D
making it different from other games of the similar genre which are 2D.
The image in the top right is a moodboard for the initial style of the game and the bottom picture
is concept art made by an artist in order to
help me visualise a more concrete version of the game.
In the final game the user can choose three different units to spawn. The first one is the Melee
Group unit which when purchased
spawns 16 small units that try to reach the enemy's portal as soon as possible. Another unit is the
Tank, who will block the path
of all units and has more health then the other units. The last unit type is the Cannon unit,
it targets the closest enemy on the path
and jumps towards them dealing a lot of damage. The player gains money every second but also when an
enemy unit is defeated.
The unit design is an Rock-Paper-Scissor RTS design. Where the Tank counter
Melee, Melee counters Cannon and
Cannon counters Tank.
The game evolved and changed a lot during development. Initially the game was supposed to be in black
and white however after
a few feedback session with peer student artists and also prototyping I decided to move away from
this theme, since when viewed from far away the units could not be told apart and was
rather confusing for the player.
Gameplay wise in the beginning the units were supposed to move in one lane from one end to the other
however in order to have more units
on the screen and make the gameplay interesting, it was changed to have a 3 lane system where the
user can decide which lane to spawn units.
In order to see how ECS architecture benefits gameplay development, complex gameplay systems have to
be build. As shown before
ECS does perform faster because of it's cache friendly design. However another aspect in game
development is workflow, or in other words,
how easy it is to create complex systems and how modular the design is.
For Graveyard Warz various gameplay systems were build in order to find out how the
development in this architecture is.
Some of the main and complex systems are:
As stated before, the game evolved a lot in terms of art and gameplay. The pictures below show what that timeline of the game looks like from left to right.
The video below displays what the game looks like in the end, with all the visual
changes,postprocessing and
all the gameplay systems mentioned above.
In this part of the report the work and research done in the four planning phases (Analysis,Design,Production,Quality)
will be shown,what were some of
the challenges faced in each phase and how were they overcome.
For planning two Trello boards are maintained, the first one is a Scrum board and the
second
one is a Log board
where all the features implemented are logged.
During the Analysis phase the focus was researching about ECS architecture and Data-Oriented design.
Many different
ways of implementing this architecture were tested and prototyped.One of the biggest challenges
during
this phase was
finding a way to store data contiguously in memory but also find a way to maintain this sets of
data.
There are many sources online explaining briefly what ECS is and why you should use it, however
there
are little to none concrete
examples on the topic. On of the best source found for the ECS was a blog by skypjack [5] which
discussed briefly a few implementation ideas and designs.
In order to implement an basic ECS architecture, research on C and C++(meta programming) had to
be
done. The most reliable source for this topic and helped a lot in this project is the book Effective
Modern C++ [6].
This phase consists of implementation of the first prototype of Feather. In order for the prototype to
be
approved it had to be faster then the Objected-Oriented design,
thus tests were performed to prove the speed of the prototype.
In the Design phase more work was put into the overall architecture of Feather. Research was
done on what features
other ECS frameworks offer and tried to implement some of them. One of the main features implemented
was the ability to query entities with a given set of components at runtime.
The Crow Engine renderer was designed and build keeping this new architecture in mind.
The last thing implemented was a prototype level which was parsed from Unity Engine where initial
gameplay systems could be tested.
The biggest challenge of this phase was making the transition to the ECS workflow and building core
engine
using ECS architecture.
In order to overcome this challenge, many versions of this systems were implemented and tested, for
example the ECS renderer was included in the engine
after it performed faster then Unity.
The Production phase consists of development of the game and its core gameplay systems.Implementing the game as visioned proved to be rather challenging
at first but after peer feedback with artists, the work proceeded much smoother.
Feather started to look more like a finished and stable ECS framework and a lot of
improvements were done based on the professional
feedback of Josh Caratelli who is a Software Engineer at
Sledgehammer Games, an
Activision Studio. One of the main added feature was the
ability to iterate components using
a ForEach query which is similar to the way Unity's ECS framework called DOTS, iterates its components.
Another big feature was the implementation of Sparse Sets which increased the performance of
Feather significantly.
The last planning phase consists of the Quality phase. The main focus of this phase was to polish the game
and make it more
visually appealing by adding fog and more post processing effects as well as designing the
level,adding more props and
tweaking gameplay values to make it more interesting.
During this phase most of the codebase was cleaned up and documented.
There were also added more code sugar features to Feather framework, like the ability to get all the
entities that fit a predefined condition
by using the Find method.
This whole website was created from scatch using HTMl,CSS and Bootstrap. It
proved to be a challenge since
it was a new experience to develop a website.
In this section the focus will be some of the improvements that could make ameliorate Feather,Crow and Graveyard Warz. These improvements were considered but were not implemented beacuse either they didn't contribute enough to the main Goal of the project or they required a lot more time to implement in a way that would be effective to the project.
The game,engine,and ECS framework built during this project helps drawing a conclusion to the main
question: What benefits does ECS architecture bring to Game and Engine development?
An overview of some of the main benefits ECS architecture brought to the development of this
project are as per below:
During these 6 months I learned a lot about data-oriented design and low level programming. One of the
things I wanted to work on myself was increasing my problem solving skills and how to approach harder
problems that
require individual research.
For this project I had to research many things in order to reach my goal and I can safely say my overall
programming and
problem solving skills have improved.
I also wanted to learn a lot not just about ECS but also about C and C++ increasing my overall knowledge
of these languages
since I had only done a few small projects using them.
In the end of the project I can gladly say I am happy with the result. I have reached my goal and in
certain areas done
more then i expected myself to do.
I would like to express my gratitude appreciation to my quality assures from the industry, Josh Caratelli from Sledgehammer games
and Merijn Vogelsang from Pillows Willow
for their feedback and professional insight.
I would also like to thank the teachers, Hans Wichman,Yvens Reboucas and Bram den Hond for their great
feedback and guidance.
Last but not least I would like to thank my peers and friends who found the time to give feedback and
ideas in order to improve project.