Trouble with Design / Starting an App from Scratch

Hi everyone.

I am having trouble with programming my application. What I am having issues with is the design / architecture of the application - i.e., how will it work, what pieces are needed, and how will those pieces fit together.

For example, if someone told me that I needed to go code a GenServer that starts an ETS table, I can do that. If someone told me to write a supervisor for that GenServer, sure, no problem. I can do all of these things in isolation, and if someone gave me a an architecture to follow, I can punch my way through it with time and effort. But I canā€™t do it myself - I canā€™t get past the design phase.

I was just wondering how more experience engineers solve this problem. Where do you start? What questions do you ask yourself when you have multiple options? How do you even get to those options?

This is not an Elixir-specific problem for me. Iā€™m what I would term an advanced beginner in both Ruby and JavaScript as well, and for those languages Iā€™d have these same issues. Iā€™m posting on this forum because I want to build my app with Elixir.

Thanks for any help folks - I know thereā€™s probably no easy answer, but guidance is appreciated.

P.S. For those of you wondering, the app I am trying to build is one for the Stockfighter.io Capture-the-Flag games.

5 Likes

I am creating application to solve a problem.
Not to create the most beautiful and perfect application.
I am trying to make it as simple as possible.
With created application I get experience and next application will be better.

I read books about programming, architecture and refactoring to improve my knowledges.

Steve McConnellā€™s Code Complete helped me to understand how to deal with application development.
I have found a lot of answers on my present questions and a lot of answers on my future questions :slight_smile:

My current plan:

  1. Start from paper - design app functions, entities, db tables. It can be raw design. You can improve it during prototype creation.
  2. Create prototype.
  3. After or during prototype creation I understand, that something can be better and I improve it or create new improved incarnation.
6 Likes

I occasionally have the same problem. It is very hard to get to the ā€˜where to startā€™ after having an idea in your head. I think it is the programmers variant of Writers Block.

This is very good advice.

What works best for me, is to first draw a diagram on paper. This might be a mindmap, or some diagram of how I think the different parts of functionality will need to communicate.

After this, there are two ways to continue:

  1. Start at the high-level, create ā€˜stubā€™ classes/modules (depending on the language) that donā€™t yet do the work internally that they might at some point, but talk to each other in a way that is more-or-less what they will become once it is finished. (Of course, in practice things will change a lot before actually being finished.)
  2. Start at a small section with a very defined, isolated behaviour. Implement this behaviour and then move on to the next, slowly building larger and larger pieces.

The most important advice, which is in line with what @dgamidov said: Keep It Simple. Do not envision something enormous and try to build it all-at-once. Instead, build the smallest thing that does something in the direction youā€™re working in, and then continue expanding on that.


I donā€™t know about the specifics of Stockfighter.io, but Iā€™d expect that youā€™d need the following systems:

  • Something that gathers information from the HTTP API.
  • Something that sends commands that your program has decided on to the HTTP API.
  • Something that decides what commands to take in what situations.

I would first try to make the part that can read information from the HTTP API. When that works, you could move on to creating the part that sends a result to the HTTP API (Which at first would be stubbed to be static, regardless of input).

After that, the part that picks smarter actions can be fleshed out.

Have a marvelous time,

~Qqwy/Wiebe-Marten

5 Likes

I always do the following:

Take pen and paper or the digital equivalent
I prefer doing both first on paper because its faster , then on draw.io for documentation for the customer.

Get a quick overview
Draw the 30 000 feet perspective of the application, what should the app do and in which order here is the short check list I do, Do not go into details as this list will change a lot during the process. draw a simple diagram based on input. This will most likely change.

  • platform(s)?
  • online offline use cases is data needed to be persisted somewhere ?
  • technology requirement (sometimes you have to use Java, somewhere)
  • What degree of user control is needed(user admin panel etc)?
  • Any security issues
  • Need/have a database ?
  • Need file Storage(local and cloud) ?
  • What kind of internal and external dependencies ( like it needs to communicate with sharepoint or fancy microsevice ABC)
  • what is the minimum this thing needs to do
  • How many users/connections directly/inderectly expected (users can be devices for IoT as well)
  • How much data expected (rough estimate that you will calculate as you go)
  • What data should be logged if any
  • what is this going to do when all done (draw up view/screens on a separate paper to)

Now you should have a rough system overview and a GUI overview that will most likely change as we go.

Describe the user interface in a wireframe
reference the GUI overview and do the following for each screen.
I take an A4 paper and split it two by drawing a line. and draw the first screen on the left hand side. For every bit of logic on the screen; name it and put it in a list on the right hand side. You should never run out of space on the right hand side. if you do, most likely the View needs to be redesigned or you should not write so explicitly what you are doing.
Do this for all the screens.

Describe the logic
From your first view start at the top of the list and se how to best implement the desired functionality.
take each functionality and split it up on its own sheet try to make groups as in Data storage/GUI/Calculations/External resource etc. and get a rough overview of what can be reused or grouped together.

Put it together
By now you should have a rough overview of everything the application should do. draw a diagram over the entire system or update the first overview you made. most likely there are new views and more blocks in the functionality overview. I tend to redo the entire thing as its just as fast.

Segregate the view Depending on the questions asked you should have a good overview over what goes where.
This is not Agile development and its not waterfall production but a healthy combination of the two I have been using this methodology for almost 20 years small and large projects.(from small services and web pages to enterprise apps with 7 million users and heavy data loads)

just to clarify its not waterfall because you should update as you go and the real world problems present them selves (there is always something)
and its not agile because you will spend a lot of time planning in the beginning.

But !
You have documented the entire process for the customer , given accurate pricing and most likely are delivering on time because you have sorted things out.

Finally , the end of this post.
The architecture will become evident while asking the questions and starting with the views. As a rule of thumb I tend to segregate as much as possible, try to keep dependencies as low as i can. and use the right language for the job.
If I were to talk pure architecture I would need specifics of your projects since there are more ways than one to do things.
However with the right approach to starting something the architecture usually reveals itself automatically.

Also I will rewrite this as a blogpost if someone is interested in me elaborating on details I did not mention that you should also put estimated cost and time in order to really blow your customer away with your professionalism.

I hope something in my ramblings are of use to you.

16 Likes

@StevenXL are you working on your own on this thing? This is rarely good, especially in design phase.

We, programmers underestimate the human factor, and we are tribal animals. I run a small consultancy, and I found over the years that it is critical not to leave single developer to design decisions at the first phase of the project. Also later, when a large changes or decisions need to be made, we tend to grab a few developers from other projects and simply talk, do some sketches, make suggestions, brainstorm.

Grab a friend or a coworker, talk him/her through. If you canā€™t get a person with your domain and/or technical knowledge, do not worry. Find someone who will listen nevertheless, even if they donā€™t make suggestions you may come to good conclusions on your own. How? Rubber duck debugging effect: by formulating a problem and putting it to words, you put yourself into a new perspective. I cannot count how many times I had a design problem, or some other programming problem, and I grabbed a coworker to explain it to them, and even when I wasnā€™t done with explaining - I already had a solution.

10 Likes

This is probably some of the best advice a programmer could get. I have a few people that I do the same with.

6 Likes

@Hoegbo: What an amazing post. I would really like to read about this in more detail: Yes! Please make a blog post about it! :slight_smile:

I also agree with @hubertlepicki. Having more perspectives is always better, and leaves you from the ā€œIf only Iā€™d thought about that soonerā€-effect.

3 Likes

You might find this blog post worthwhile reading. TL; DR trying to make things perfect right from the start is a great recipe for procrastination. Make a startā€“even an imperfect oneā€“and push through.

ā€“
Onorio

4 Likes

Hi everyone - I wanted to drop a quick note tonight because, though I havenā€™t had the time to go over the responses, I do appreciate everyoneā€™s advice. I will come back when I have some more time (likely over the weekend) for a more thorough response. Until then, thank you all for the advice.

3 Likes

I will start refining and rewriting and make a blog on Medium. Expect it to be ready next Tuesday most likely

7 Likes

Hi Hoegbo,

No rush, but wanted to make sure I didnā€™t miss your post. Is it out yet?

2 Likes

@Qqwy This is great advice. Thank you. I do think that going from the very high-level (I need piece A, and piece B), and then implementing the minimum for each piece, is a great idea.

4 Likes

Hi, great write-up.
Please do share the link to your blog post on medium once itā€™s done.

Thanks.

1 Like

Blog post is coming, just had to prioritize some other stuff first.

3 Likes

Discussing the design with other people is always good - making yourself explain it to others should help it become clearer to you, and / or it will point out to you what it is that you donā€™t understand.
The other thing is donā€™t be afraid to change things, or even start again from scratch, if things arenā€™t working. We all make mistakes, and the process of designing / working on the project should reflect that.

2 Likes

Thank you @riverrun. This is all very helpful. Yes I need a group (aka this forum!) to get the concepts out of my head and organized on paper, so to speak.

3 Likes