Materialization of a Mine Field

Source code, basically, is the materialization of an idea, and the more complex the idea is the more you see an explosion of possible ways to materialize it.

When in the process of materializing an idea (coding), multiple competing constraints emerge and try to influence the final materialized form.
Examples of constraints are performance, readability, maintainability, extensibility, time, elegance, typing economization and probably others as well.

This happens even when writing something as simple as the declaration of the function.
inline? virtual? pass arguments by reference? which arguments to pass? how to name the function, how to name the arguments, implement inside the header file?
each of those decisions can satisfy or break constraints and there is most often no way to satisfy all of them, and all we wanted is to declare a simple function.
One might thank that choosing which arguments to pass is obvious, but many times it is not, there are many variations and optional arguments that are handled and the decision has to be made to code many 'interfaces' to the function accepting slightly different arguments, this might satisfy readability constraints, but violate maintainability ones as I will in some examples at the end.

Moreover, the more the idea materializes, the more important 'piping' becomes, and the exact declarations start to become a big role, since nice 'piping' is also a very valid constraint. This means that it is not even possible to design beforehand a 'perfect' materialization, because the constraints we mention operate on the most microscopic level, and this would mean that the perfect design would need to go down to the lowest level, but then the design is the same as the actual coding, which beats the purpose of a high level design.

So all these decisions have to happen at the time of materialization (coding) and as soon as one form is chosen, some constraints are violated, this cannot be avoided, following the great wisdom of Japanese Anime Black Lagoon II: 'when you choose, you loose something', this is in fact unavoidable.
It is a really annoying fact, the better one wants to satisfy all constraints the more impossible it gets. This is a usual problem manifesting itself everywhere really, science, philosophy, every day life, everywhere really.  One might say that achieving a balanced form which satisfies most of the chosen constraints for the current context is optimal, and yes this is what we usually strive for, but it is still, a compromise and we still 'loose something'.

The important thing however when it comes to coding is to be AWARE of the chosen constraints and of any compromises that were made, and the consequences of all the forms that were chosen.
Awareness puts us in control instead of being unknowingly controlled by constraints sitting in our sub-conscious mind (readability freak, micro-optimization freak, code elegance freak, ...) and allows us to walk through the mine field with a headlight and a baseball bat, instead of blindly stepping on the mines totally unarmed.

In the examples, and yes they are almost ridiculous, I chose very simple functions, and one might think that this is exaggerated, but one can notice that at every example decision  some constraint were broken in the favor or others and hence a compromise was made.

Example1:

    1. float dist(const Vector3 &a, const Vector3 &b);
   
    nice and short, however, 'distance' is more precise, more readable, but needs more typing ...

    2. float distance(const Vector3 &a, const Vector3 &b); //more readable

    but it is more performant to determine the squared distance and it is all whats needed     in many cases

    float distanceSquared(const Vector3 &a, const Vector3 &b);

    this is too much typing

    float distSq(const Vector3 &a, const Vector3 &b);

    this is too cryptic

    float distanceSq(const Vector3 &a, const Vector3 &b);

    but how to implement this?

    inline float distance(const Vector3 &a, const Vector3 &b) { return              sqrtf(distanceSq(a, b); }

    but makes the header file look dirty, let the compiler do the inline optimization for us

    float distanceSq(const Vector3 &a, const Vector3 &b); //implemented inside .cpp

Example2:

    1. float distance(const Vector3& segA, const Vector3& segB, const Vector3& pt);
   
    segA? what's that? too cryptic
   
    2. float distance(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point);

    that cost more typing, but ok, now I have an Idea why don't we write it like this?

    3. float distance(const Segment& segment, const Vector3& point);

    let's make this cleaner and move it into the Segment class.

    3. float Segment::distance(const Vector3& point);
    
    but sometimes the code needs to compute the distance directly from 2 points, without having a segment,
    but we still want this version, saves typing when dealing with segments.
   
   4. inline float distance(const Segment& segment, const Vector3& point) { ... }
        or
    inline float Segment::distance(const Segment& segment, const Vector3& point) { ... }

    now we need that squared version again

    5. float distanceSq(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point);

    but now we need to maintain 2 functions that do the same because of decision 3, so now we need to add

    6. inline float distanceSq(const Segment& segment, const Vector3& point) { ... }

    that's all!!, but no wait, many times, we need to extract the closest point at the same time, ok, add

    5. float distanceSq(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point, ector3& closestPt);

    but wait, we can merge the 2 to have less maintenance into

    6. float distanceSq(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point, Vector3* pClosestPt = NULL);

    but that costs the callers that don't need to calculate the closest point some performance.
    doesn't matter, this is good enough.
    there is one more thing though sometimes all we need is the resulting interpolation value 'u', 
    specially when using this implementation:

        float u =  ((point.x - segmentA.x)*(segmentB.x - segmentA.x) + (point.y - segmentA.y)*(segmentB.y - segmentA.y) + (point.z - segmentA.z)*(segmentB.z - segmentA.z))
                        / ((segmentB.x - segmentA.x)*(segmentB.x - segmentA.x) + (segmentB.y - segmentA.y)*(segmentB.y - segmentA.y) + (segmentB.z - segmentA.z)*(segmentB.z -                     segmentA.z));
        interpolate using u

    no need to calculate the exact closest point... ok

    7. float distanceSq(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point, float& u);

    or maybe

    8. float distanceSq(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point, float* pU = NULL);

    or even

    8. float distanceSq(const Vector3& segmentPointA, const Vector3& segmentPointB, const Vector3& point, float* pU = NULL, Vector3* pClosestPt = NULL);
    
    nah, too much checking NULL pointers, so better write many versions, but that's         more annoying to maintain

    9. ......................you get the point, and those are only declaration of simple functions................






'Experienced' AI

Since some time, and you would know it if you have been following our blog, we are at the stage of working on WOF's AI since the basic gameplay mechanics are finally in place.

We had a first 'flirt with AI' iteration and posted videos of the results.
We are now at the second iteration which should already produce some useful fun-playable behavior. I decided to start with the goalie. after some coding and much thinking I came to the conclusion that I will probably have to include some AI 'experience' that the AI footballers' brains will be able to use.
The reasons for this are simple: without this, a goalie's brain would be running full steam the whole time trying to estimate and reestimate the same things. things that real life goalies know by 'experience' because when a footballer steps in the pitch ... he has years of playing experience and training behind him ... can this be simply discarded? well yes and no.

If we choose to give the footballers no experience, they will need much more processing power to compute potential ball positions after shots, potential shots to make, how dangerous a shot could be given the situation..... it's possible but both expensive in terms of processing power and therefore hurting game performance, too 'machinistic' in terms of precision ...

the footballer would be able to estimate to with exact precision if per example the goalie can catch a ball if it was shot in some way, but also and on a finer level probably too predictable for the human player who would then find a situation where he could score once and repeat it forever because the AI would never try to fix whatever went wrong with their estimation.

Now how on earth do we give our AI experience?
A neural network brain comes to mind ... a brain that knows nothing at first and is trained through a huge number of iterations until it has learned something... but nahhhh its too unpredictable, fragile, uncontrollable, unmoddable and is better left as an exercise to academics. the goal here is a fun game ...

I will try to use some time at load time, before the match starts, and use it to make the AI learn a bit about what shots can be made using the current match situation (ball paramters, pitch parameters, player characteristics) and they will then be able to use this as 'experience' and in a totally human way act based on approximations and even try to fine those approximations when they find they didn't work.

its more or less like sending the team to the pitch for a short training before the game, so that they get their brains in-tune with the match's parameters and so that u don't play against a totally 'unexperienced machine precise hard coded AI' team.

I hope it will work well. this might again take some time ... but it is a topic of huge importance!

so until the next vids, stay tuned! icon_mrgreen


The world is catching up ...

(WARNING: yes you guessed it! this is a rant)

Again and again,

I have to justify the in-existence of a release schedule for WOF to almost everybody I know, from friends who just care about me to friends who are software engineers themselves and who keep on wisely preaching me on how the mere existence of a schedule would auto-magically accelerate the development process.

I usually reply to those with proofs of the too many to mention 'professional' and super planned projects which end up being discarded with millions of $'s of losses because in the end, the schedule was not met anyway ...

Of course, if I had employees who's primary goal was to produce as few sweat drops as possible while still getting their salary at the end of the month then yes, schedules and deadlines and manages on top of managers hunting each other with dates would have been the way to go. And that is probably why this is needed in dinosaur sized companies where passion has no place, and people do the work with the salary as the ultimate goal, except maybe for the arrogant narcistic managers, sitting at the top, getting their fat salaries and bonuses, having a good life, and then just for the fun of it trying to push their company forward to show off in front of their friend managers, then preaching their employees about passion, with their own passion coming from the wrong place.

BUT we are an INDEPENDENT studio, we sweat and torment our brains every day and we do it because we want to, we need to and we want to make our dream game, and try to make it financially successful just to be able to repeat and re-repeat that. That is why for us, a schedule is just a hurdle in the way of a perfect game. It's existence will not make us work faster or better, because we already do that and only that!

In any case, it seems the world of dinosaurs is catching up and realizing that actually, its us who have been doing it right from the beginning and that they need to revise the way they develop: using a manager and a schedule ... suddenly it's starting to become hip to do things our way! yes ... agile development is the new hip way to do it, http://www.agilemanifesto.org/, and look there! IBM wakes up thinks so as well ... http://www.infoworld.com/archives/emailPrint.jsp?R=printThis&A=/article/08/03/04/IBM-promotes-agile-development_1.html stange? not to us!

Our and your dream game will be done when it's done! (because that's the only path to a dream game).

Fivemagics*


Bug hunting and AI..

The last few weeks have been spent on gathering feedback from our devoted and patient testers - and fixing the errors.

Our testers have been very good at delivering information that we can use and willing to do some tests and discovery on their own. We're very pleased with the help we're getting, and we hope you'll continue to be patient with us going forward! =)

It's also satisfying that most reports have to do with technical aspects - incompatibility with certain graphics cards, or getting the game to run on the Vista platform - and only very few design flaw or implementation errors. Below is an example of how things can go wrong when a graphic card is not fully supported - but this have been fixed now btw.:

Most bugs have been sorted, and Fivemagics have begun on actual implementation of our AI concepts. It's already great for me to see the footballers run around and from here on it will only get better. In the video below you can see a first glimpse of some really basic stuff in action; footballers running to positions, and a footballer anticipating where the ball will land and running to that position and intercepting the ball.

As some might have noticed the stadium is still incomplete. We have decided to upgrade our art-pipeline, which means using another exporter programme for the 3D files. This exercise has been postponed until the AI is further underway though, so you will not see an update in the surrounding graphics for some time.. Maybe that only hurts my pride - but we're sticking to our initial plans and putting gameplay over eye-candy.


WOF gets ImageMagick'al

From now on, powered by the combination of a scripting language (Squirrel) and an Image Manipulation Library (ImageMagick),
WOF enables us to do funky things to it's image based assets and that, without changing one bit of WOF's code. (think modding ...) icon_twisted

I realy don't have time to waste, crunching on our beloved WOF, but I simply had to have some fun 'modding' our unfinished test stadium, and here are the results. (click to enlarge)

Reference screenie (the way the 'stadium' ususally looks like)


Auto-Generated Boards and Saturated Green Pitch
(this was simple text generation, more complex boards with images and all kinds of things are also possible)


Close Up icon_mrgreen

Yes it does! or more accurately, it will! icon_biggrin



Save! Goal!

Here's a little example of what happens when Fivemagics combines the ball physics and the script for animation and the ball and footballer collisions...

How cool is that! icon_cool


Robust collisions..

Here's a video captured by Fivemagics showing the footballer v footballer and footballer v environment collisions. He's a man of few words, and I'll probably get some stick for quoting him, but here's a rare opportunity to hear his own words about it:

'smooth action, while keeping totally robust non-penetration and footballer blocking (for tactical purposes) and still allowing for dramatic and funky tackles while keeping totally robust non-penetration with random world geometry'

We weren't sure whether to release this clip, but we thought why not. It is not polished in any way and only made to show the collisions in action - its not a gameplay movie! You'll see footballers NOT going through objects when running or sliding etc. and not getting stuck on objects either. Sounds simple but it is anything but.. =)

At the very end you can catch a glimpse of the early work on the keeper movement using a very clever custom script.. more to come later on this.




Fivemagics does it again!

- Amazes me that is!

Here's a movie showing the very solid ball collisions in WOF. The ball can go anywhere in the stadium and it will react properly to any objects.. It looks great, and the sound fits perfectly too - in my humble opinion! =)

The only trouble is that the stadium looks very incomplete, so I better stop writing and get back to doing some work.. =)




Camera ... evolved

Here is a small comparison of how the camera system recently evolved ... its still pretty basic ... anyway heres the old and the new setups:
Old Setup VS. New Setup


your wheels and my car, again...

http://www.gamedev.net/community/forums/topic.asp?topic_id=440434

a point of view related to mine, it seems is the one by "bootstrap"

"Excellent! Unlike 99% of people (including programmers) these days, you ..."


Some boring school math





The Coder Poet ...

There is no perfection, only beauty


Unusable Chameleon

Amazing!!! the huge number of ways one can write the simplest of classes, and the huge number of 'seemingly' good ways that are in fact very limiting ...

basically, coding is choosing from the infinity of theoretical 'code solutions' one and materializing the chosen one into high level language text. imagine the sheer number of equally good or better solutions u have left out, how good solution is can be determined by current requirements but also depends n the future, so its an optimization problem, how far and how hard will u look into the future (which could also be a waste if done out of proprtions) to choose thet code u will materialize? how detailed will ur thinking be, to choose between the equally infinite amount of detailed ways to materialize the code which would make it best usable? u could even choose to code mulitple versions, (inline / not inline controlled by a #define per exmaple, with more or less control), if u do that how willu optimize the effort of trying to make those versions as similar as possible ( for obvious reasons) while as different as possible (fucntionality and usefullness wize) ... its all a nice huge optimization problem.


high on c++ templates

QUOTE:
template <class T> struct Ptr {...

QUOTE:
template <typename _T, typename _ManagerT, typename DestroyTraitsT>
struct ManagedPtrT : Ptr<_T> {
...
//struct defined inside ManagedPtrT
struct Auto : Ptr<_T> {
SoftPtr<_ManagerT> manager;
... }

QUOTE:
template <class T>
struct ManagedPtr : WE::ManagedPtrT<T, Device,
ManagedPtrDestroyTraits<T> > {

QUOTE:
Device::ManagedPtr<Device::FFVertexDef>::Auto vertexDef;
no comments. icon_twisted


Code Paranoia part I ...

I just added the following file "WERendererD3D9ProgramDataTypeHelper.h" to our engine. that is pretty sick if u ask me, but as much as i hate it, I am too paranoid to rename it to anything shorter ... snif ... and its not the only one.




Variable Arguments ?

yes, I am sick of inconsistent support for variable argument functions, and the annoying fact that in many cases u need 2 version of variable argument fucntions (one with (...) and another with (va_list)).

I only ever needed var arg functions for string operations, parsing and logging. so again I got rid of using (...) once and for all ( ? ).

The results look like this:

QUOTE:

//standard varibale arg
fct(L"Hello %f, %d", f, i);

//new way (for logging per example)
P_EL args=("Hello ", f, ', ', i, P_END());
fct(args);

//new way (parsing example)
P_EL args=("Hello ", &f, ', ', &i, P_END());
fct("Hello -2.6,66", args);

Thats as comfortable as i was able to make it, and i like it ... u can of course also setup a dynamic P_EL array at runtime.


seemingly 'simple' strings

Ok, Im done with finalizing strings in the engine, yes yes we had working strings since the beginning, but everything always evolves, and code does as well, we started off with simple 'char*' strings, so no internationalization (badly needed for WOF), the next version support UNICODE 'wchar_t*' which i assumed was the end of all string things... but it seems using only the documentation of 'so called' 'unicode equivalents' of simple 'char*'string functions never mentions that in fact a 'real' UTF16 char is variable width!!! Short Look at sample headaches/documentation pitfalls

so back to the design table (hopefully for the last time for strings).

in the end i decided to go for UTF-8, has all the advantages I need, and in thie way I can also easily move text files and binary files (containing text) happily between OS's no problems at all.

it works quite well, I also added some goodies for 'OS' independence, Strings can be converted (or not when not needed but still transparently) to a 'NativeString' which is used to pass to native functions like windows functions which have ANSI/Unicode variants and so on .... this makes the whole OS dependence nicer (it is already nice and hidden away in nice 'Native' Intrface classes, but now Strings can be passed in and out nicely).

The functions are not optimized, but they will only be heavilu used at loading time and should not be used in performance critical parts anyway (comparing strings? NONO).

Bonus: Another thing I like is that the string code is now more general becuase it doesnt treat each 'char' as a 'glyph' so now switching formats (which most probably wont happen anyway) would be more or less painless, specially since I already needed to write conversion between ANSI-UTF8-UTF16 to support char*, wchar_t* and the internally used UTF8.

Bonus: no more const static char* kBla = "bla" thrown here and there, these now become const static CtString kBla("bla"), this is converted of course into the internal UTF8 format, well probably not a big difference but I like it more...

Bonus: In code there were places where temp string are passed around a lot specially while loading, a 'BuffString' was used for this which until now simply typedef'd to a normal string, but now i added StringMemAllocator supports, so now u have real Buffer strings (when u want them) which reuse memory and save a lot of free/alloc calls.

As for string length including/excluding the ending NULL character, I decided to include it, after all is IS a character in the string, be it special or not. and i added a 'bool lengthIncludesEndingNULL()' function which shoud be used when manipulating strings with no assumptions. String code doesnt have to be super fast, only super usable, at least for the projects in-sight.

Another indirect bonus is that we use tinyXML (currently) which supports UTF8 and we couldnt use this support (for player names per example) until now.




const& , overflow and c++

Part 1

Constant references, instead of passing by value is it worth it (for 32 bit and smaller data types of course) ? tough to say, in most cases it surely isnt slower than passing by value, or maybe it is when the value u are passing is smaller than a reference (pointer) ? like passing a char is 1 byte but a const reference to a char is 4 byte (on 32 bit system). I really need to time it, but i guess it laso depends on compiler/processor. but for now i seem to be passing values by const& wherever possible in my code and cant seem to be able to stop, the only thing is function calls become too long ...

Part X

overflow, I started using bytes as indexes in a part of the code to save space, dont know if its really worth it, but i think it is, the problem now is letting the code warn u with overflow, so u start hacking checks using temporary variables and casts to see if the value changed sign/overflowed and the like, wont go into details, but i wonder why there isnt any overflow support in c/c++ itself, pretty annoying if u ask me ...




Your wheel doesnt fit my car ...

Every now and then I rememeber that I decided at some point not to use STL for WOF, and then I start to think about how 'unproffersional' that is and start hearing voices telling me to switch to STL.
In fact I think my 1st use of STL was in 2001, sitting in my office, searching for a library that would give me good strings/containers and the like. When i first found STL and started reading about it I was pretty impressed really...
but enough about history ...

there are countless forum discussions and threads discussing code reuse and 'reinventing the wheel'

from coders hanging anybody not using STL
to coders beinbg shocked that DOOM3 doesnt use STL
to longer ...     discussions

but today i decided to change a bit of code that handles string parsing, and instead of using plain old C sscanf and friends, to use the new gr8 genius C++ iostream while this is not 'exactly' STL but its related ...

anyway, i was happily recoding this 'evil' 'unprofessional' C style code, and scanning string using >> and all that fun stuff, its fun cause it also supports locales and much much more thing i dont know about (im not sure this is a good thing though)

but my dreams came to an end when after succesfully scanning floats, my code sudeenly refused to scan old simple integers.

and i m saying my code because of course its my code thats doing sthg wrong, and surely not 'std' so i spent 1 hour looking at all those very improbable causes in my code, until i gave up, and in an evil split second of unprofessional thinking, I typed this in google:
iostream problem reading int comma
which found nice matches, and after the usual journey in google I found:
link
link
link

and so on ...
all these troubles for reading an int? yes of course I could fix it, there are workarounds, there are other STL libraries to use, and regression tests to make sure they work.

but all i needed is to read an int, I think i had code to do this 10 years ago when i still had no internet and was a total noob and was proud of a function that converts strings to integers, and even floats! :P
sure std has locales and this is why this messes up, but maybe this is not how i want to use locales, i dont need a cannon to kill a fly, sure STL is tons of code that can do anything, but the things i need from STL i coded for WOF in a couple of days and worked reliably with no problems until today.
so I dont want to be switching STL libraries for my code to work, I dont want to have headaches every time somebody changes something somewhere, I dont want to work around libraries that theoretically are 'pieces of coding art' until u need to do something practical with them.

just found this:
STL Guru Interview

QUOTE:
In other words, I realized that a parallel reduction algorithm is associated with a semigroup structure type. That is the fundamental point: algorithms are defined on algebraic structures. It took me another couple of years to realize that you have to extend the notion of structure by adding complexity requirements to regular axioms. And than it took 15 years to make it work. (I am still not sure that I have been successful in getting the point across to anybody outside the small circle of my friends.) I believe that iterator theories are as central to Computer Science as theories of rings or Banach spaces are central to Mathematics. Every time I would look at an algorithm I would try to find a structure on which it is defined. So what I wanted to do was to describe algorithms generically. That's what I like to do. I can spend a month working on a well known algorithm trying to find its generic representation. So far, I have been singularly unsuccessful in explaining to people that this is an important activity. But, somehow, the result of the activity - STL - became quite successful.

I understand all this! honest! I DO have a BE in Computer and Communications Engineering, and its gr8, artistic! nice and kewl, takes it all to a new level! but there was sthg wrong with the implementation i just used reading an int ...
QUOTE:
Yes. STL is not object oriented. I think that object orientedness is almost as much of a hoax as Artificial Intelligence

yes it is, but the same goes for code reuse and over-engineered floating in their own heaven of pure theory libraries ...

I actually could write much much better comments about them, but im tired and i have better things to do like workgin on WOF.
One point I still want to make is the horrible naming convention of iostream: its really chaotic and unintuitive, but i guess iostream is allowed to do this, to do things u would be hanged for if u did urself. (pubimbue, snextc, sbumpc), maybe to support 8 bit operating systems, but this is really not my problem ... I wont expand on this either although I would like to.
so as a conclusion, i ll stick to my own wheels until the next time i have STL voices in my head.
and just for the record, i do use other libraries so its really not a 'i dont use anything i didnt write syndrome'

end of bla bla ...




Page :  1