File Formats and thoughts on design

Oliver Marsh —
You hear the word designer a lot these days. Whether its a a ui designer, graphic designer or a level designer. In part of my mind I associate 'designer' with a more arty type like a graphics designer. But I like design and I'm not particularly an arty type. I like the idea of making something more functional. The engineer definition of design. But I have part of me that sees and believes every human activity is design. The chef designs food. A mechanic designs solutions to car problems. The physicist designs theories. If I think about it, the broader definition of design is understanding and using elements of a domain in ways that solve problems whether they be aesthetic or functional. We operate in a space of concepts.

The elements of physics are the law of conservation, calculus, theorems etc. and they are used to design theories. For a programmer the elements are 'for' loops, 'while' loops, caches, data. The spaces that we solve in also have different levels. If we are doing graphics programming, vertex buffers, over-drawing become elements in the space that we are solving in.

In this movie they had an analogy of there are all these universes existing at once (the spaces we operate in), and we are like a radio that can tune into a station. The deeper we know a field, the multitude of different wavelengths exist for that universe that we can tune in to. A capenter sees the grain of the wood, understands how the wood will bend and cut with different implements, knows what strength the wood can handle. But an inexperienced wood worker only sees wood.

What this is all about is game design. A player comes to a game and solves problems within a space that they are presented. The space they solve in is crafted by the game designer. So as the game designer, the space I'm working within have elements like platforms, gravity, doors, game mechanics. So as I approach the game, I have two spaces I'm in. Programmer space, in which I feel like an engineer. Then the game design space, where I drag around objects on the screen, piecing together levels. Although there may be some overlap, I'm trying to just think with the unique elements in each space.

So in order to experiment and explore this space, I need a very strong save/load ability that allows iterative workflow, and that I'm not scared to use (because it will crash or corrupt the data). Previously I implemented a save/load data system that created just one file that contained all the entities, and the data was set out like a relational database. I found this wasn't the best solution. All the values were dependent on their order in the file, so if something went wrong (a value was accidentally deleted), I wouldn't know which number corresponded to which entity attribute. Also for enum types I just set the number, not the corresponding string, which doesn't jell well when you want to change the data in the file, look at the data and if I change the order of the enum, the data is invalid.

So coming at it this time I had a much higher bar. I first did the save data. At first I wanted to have a binary format, as it was suggested by Mr 4th dimension https://top.handmade.network/blog...s_on_cross_cutting_concerns#14759. But in the end I decided to go for a JSON style text format. Furthermore each entity is it's own text file, so a 'level' corresponds to a folder of entity txt files.

Each entity file looks like this, where the first object has attributes unique to that type of entity, and commons are attributes that all entities have:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
Door: {
	partnerId: 2;
}
Commons: {
	id: 1;
	name: "player";
	flags: 251;
	type: "ENT_TYPE_DEFAULT";
	position: 1.000000 2.001000 -1.000000;
	renderPosOffset: 0.000000 0.000000 0.000000;
	dim: 1.000000 1.000000 1.000000;
	renderScale: 1.000000 1.000000 1.000000;
	shading: 1.000000 1.000000 1.000000 1.000000;
	texture: "door1.png";
	inverseWeight: 1.000000;
	animation: "knight animation";

}


For loading I followed the header modules engine style and made a portable lex.h file. It just creates tokens for a given char stream and has easy methods to move through them. At first I though maybe I could then create an Abstract Syntax Tree and treat it more like an arbitrary language. But I didn't end up doing this, as it seemed alot of work so I just treated the file as declarative data with implicit types.

Overall I think this is a much stronger file system than before. It will allow me to experiment with gameplay and allow me to edit offline easily. Maybe for the shipped game Ill move to a binary style file format that will be quicker to load at runtime, but for now this works great.

Any comments, suggestions or thoughts please leave below!

Oliver

NOTE: This philosophy further expands to we only know something in how it relates to other things. We can't know something absolutely. I think this is similar to Thomas Kuhn's 'The Structure of Scientific Revolutions' where a space is a paradigm and we solve puzzles (in this case design in that space) within that paridgm.

Comments

Leave a Comment
Simon Anciaux
About the file format, why did you chose to use a text format ? It looks like you have lots of things that would be easier to modify using an in-game editor than to modify by hand in text.

The title is "On Design" but it seems to me that the article is about file format or serialization.
...I found this wan't the best solution. One was that all the values were implicit in there meaning based on their ordering...
There is a typo: "this wan't" should be "it wasn't". And "in there meaning" should be "in their meaning". The second sentence is hard to understand. I believe you meant something like: All the values were dependent on their order in the file. I'm not a native English speaker so I might be wrong.

The design part of the post wasn't, in my opinion, relevant or helpful to the rest of the article. I would resume it to: Design = Solving problem with available tools. It may be right, but it doesn't help in any way to know that.

Is this still Mind Man or is it another game ?
Oliver Marsh
Edited by Oliver Marsh on
@mrmixer
Thanks for the comment. I updated those spelling mistakes and the sentence. Sorry if it is a bit muddled, I'm not always the best writer.

Yes I think the design part of the post does sum up to
Design = Solving problem with available tools
It was a bit of a ramble, it was just to show the motivation behind making a good file system. I probably should have just stuck to talking about the file system, instead of the design part as well.

Regarding why I chose the text format. I think the main points were:

1. I wanted to see what the data was. But as you said an in-game editor would work or a binary to text convertor done offline as suggested by Mr 4th dimension (in the Top blog post).

2. I wanted it to be future compatible no matter what. I think this was the main reason, but thinking about it more probably isn't such a big problem. I thought if I decoupled the data from the game data reader (it didn't expect a particular layout), it would increase the life span of the data. Versions take care of this in binary formats from what I understand. But I couldn't see the whole picture with the binary format, so I just went with the text format.

3. I had experience parsing text formats so was drawn to this option out of ease. It has the downside of having a parsing layer between the file-data and the game-data.

I think I'll try do a binary file format path and see what works out the best option.

The game isn't mind-man at the moment, mostly experimenting with game-code. It is a new code base, and I'm going about things a bit differently. I hope it will either be a new game in the spirit of mind-man or I'll it will come back to being the original AI concept of mind-man.

Leave a Comment