Idea: CBOT Editor

[Simbax]

We all have our favourite text editors, but there is no real support for Colobot level editing in any of the common text editors. There were/are some attempts to build specific tools, like @tomaszkax86 's map and relief editor or @adiblol 's tools. But what I am dreaming about, is a fully integrated environment for creating Colobot content. So, here's a basic idea:

Code:

# CBOT Editor #

Text editor created for Colobot developers and modders.

# Goals #

* Support for scene files, SatCom help files and CBOT source code
* Syntax highlighting
* Auto-completion
* Snippets
* Project view for standard data directory and for levels
* Live preview of SatCom files
* Support for translations editing
* Compiling CBOT
* Integration with the future level editor
* Integration with Colobot (e.g. running a level by a keyboard shortcut)
* Integration with version system control (e.g. git)

Generally, this whole Editor is supposed to let developer/modder to open data or level folder as a project and work in one, comfortable tool. Including, of course, necessary external apps like the game itself, map editor, image editing program and so on (there should be an easy way to open some of them directly from the Editor).

Notes:

• I think it can be written in Python/Tkinter (edit: we could also use wxPython, but that would mean need to install a dependency), so it will be easy to run (end-users will just have to install Python and download the source code) and we won't have to maintain any releases, compilations etc. (We can, however, create binary packages, if we really want). But you may come up with a better option. Frankly, I am very subjective here, because I just love Python, but there would be so many benefits. This language is very easy to learn, so this might mean more contributors. We would have a much greater speed of development than for example in C++, because of Python/Tkinter simplicity. And, there are ways to include C/C++ code (or even Java, C#, and similar), it would probably make integration with external tools way easier (especially CBOT compiling).
  
• CBOT compiling... It means checking if the program will run, like it works in game. Runtime of course must be tested in the game. It will probably be possible only after CBOT is rewritten.
  
• Integration with Colobot... I mean a special command line parameters or something like that, so it doesn't load all intros etc. and just jump directly into the level.
  
• Rendering SatCom files shouldn't be a problem, at the beginning it can be a completely different rendering than in-game, if it will be too hard to use the existing rendering logic from the game.
  
• Integration with a level editor... I am thinking here about something like Unity/MonoDevelop. Editing scenes and some other things in the level editor (probably @tomaszkax86 's?), and editing scripts and help files in the external editor (the Editor by default, of course). Plus the ability to edit the scene file by hand, if for someone it will be more convenient or there is no new function built in the level editor etc.
  

So, what do you think? Does anything of this make any sense? Is this project too unrealistic for now (or ever)? Would you even use this editor?

[tomangelo]

If Cbot will be a separate library, then we could link the editor to library, so we'll have only one CBOT code to develop. There should be a possibility to link C++ libs to Python application.

[krzys_h]

I'm thinking this would better fit as a part of level editor than a separate project. We don't have enough power to keep stable development of the game itself, a side project like that would distract us too much.
BTW, it's already possible to jump directly to the level after starting game and exit after it's finished, -runscene parameter, e.g. -runscene missions103. You can also make the game exit after the level is loaded (just to check syntax) with -scenetest.

[Simbax]

I don't want to open a whole level editor just to edit some SatCom files or translations.

You may be right about distraction.

I want to know if I can at least begin it as a personal project, even if it won't be developed by the whole TerranovaTeam. I mean, if anybody would use it, even if it had only, let's say, 10 percent of it's goal features? We'll be doing a lot of work with the levels in the future and I think it's a good time to finally have some tools, which will speed things up.

[tomaszkax86]

I think this is a good idea. General purpose development tool could be very useful, especially for us when we start working on Colobot 2. Although I'd say it will be difficult to make one big project right away. Say we make some smaller programs that can do single thing good and then we try to integrate them into one big program. One problem is that some features will change so this project needs to be kept up to date.

[krzys_h]

I don't really see a big diffrence in opening level editor or specialized text editor.

Instead of making a whole text editor from scratch, I'd rather go with something like a module for atom.io (if we really want it). This way you have some things like autocompletion or syntax coloring ready and only need to write Colobot-specific rules for them.

[Simbax]

I was doing something like that, but the effect was not really worth it.

1. Every editor handles these differently. Maintaining configuration files for many 3rd party editors? They would lost their purpose very fast, as the game changes often.
2. They doesn't give that much freedom. I can't make a live preview for SatCom files in a second panel or check if links work without opening the game or many other things. For example, gtksourceview default styles doesn't give that much options (specifying more than a half things as "def:type" or "deftatement"... what's the point of highlighting really if nearly everything has the same colour?).
3. There is often no easy way for detection. I would need to set the syntax highlighting and maybe even special style every time I wanted to edit something related to Colobot.
4. How to even work with translations? I would want to see the original English file and with one click view the according line in *.po files.

Our formatting syntaxes are not very common. It would be a totally different story if we had some sort of script language as CBOT, or HTML as SatCom and so on, but we don't. Do you know how many times I had to rerun the game because of missing semicolon, newline, wrong indentation and so on? Of course, all of these problem could be solved by making separate modules for some 3rd party programs and linking them together, but I don't like the idea of downloading, installing, configuring etc. 10 applications just to have an unstable and unconnected directly substitute of Colobot development environment.

And note that these things are not something that is unsolvable or doesn't let the work be done. But all these little problems culminated are slowing down the work. What I want to achieve with this editor (or set of editors, whatever) is simplicity, which by definition results in faster results. Something like "oh, I need to edit some CBOT AI, let's open the CBOT editor!" instead of "oh, I need to edit some CBOT AI, let's open an editor... wait, where is my syntax highlighting, oh, I need to set tabulators instead of spaces, hmm, how to use that instruction, well, maybe I will run the game to check, oh man, maybe there is a better editor out there, let's find something...".

[krzys_h]

1. You wanted to have one editor anyway, why would we keep multiple configs for different ones?
2. Atom.io is designed to be very flexible, I'm 99% sure there is a way to add more colors
3. That IS a problem but there is nothing we can do about that, the best we could do anyway is to make the editor guess what type of file is that or change extensions to be something custom
4. From what I understand, the gettext architecture is not designed to work like that, but exactly the opposite. You are supposed to edit only the one *.po file so you don't have to even open original files, because you have the original text in the line above. It also makes sure the general structure of the file is the same across all the languages (you cannot freely modify it).

As for our non standard syntax, like I said, atom.io is very flexible. From what I've seen, you don't specify the keywords to highlight, you actually specify regexps, and configure a color for each match group. So you can, for example set an regexp on \l;something\u target_file; and have the link text, target and formatting tags each colored differently but only if the WHOLE thing is correct.

EDIT: Oh, and I'm pretty sure you could make SatCom preview render in atom.io if you have time to learn the API

[Simbax]

You really like this Atom, don't you? Last time I used it, it didn't have a good impression on me, but it was in beta, maybe I should give it another chance.

[krzys_h]

I just love the concept of it being so hackable

[Simbax]

Maybe we should change the extensions. It would solve at least one of the problems.

My suggestions: *.cbot, *.satcom, *.cbotscene. Yeah, I know, they are not very creative, but why complicate things too much? They are unique and readable.

[tomaszkax86]

Scene files can be just *.scene. There's no need for creativity here.

In the future we could add full support for reading data straight from .zip files instead of subdirectories. We could add all sorts of metadata to a .zip file and keep everything in one place. They could also serve a purpose of project files. A project could represent entire mod content, one planet or a single user level, depending on needs. -runscene could be modified to accept .zip files to directly test levels.

[krzys_h]

Reading .zip files is already possible with PHYSFS: -mod path/to/file.zip. That can be also an unpacked directory if you prefer. You probably want to use that together win -runscene custom101.

[Simbax]

I was playing around with Atom today. It is not very well documented for "hackers", for example there is very brief explanation of creating grammar packages and there is no list of standard classes which should be in every theme and I had a lot of problems with nesting tags, but I managed to figure it out somehow basing on other packages and here is the result:



Atom is pretty cool and useful, I must admit. I will probably do a whole package for Colobot instead of separate ones for CBOT, SatCom etc. and then I will create a public repository on GitHub, which may be added to ours later on.

For now, I give you what I have done today in attachment. It currently works the best with One Dark theme. One Light for some reason gives dark letters on dark background for \s; and other built-in themes have less colors (probably different CSS classes, I have no idea how to choose them reasonably, the only way I can think of is to make a separate style, but that would force users to use only this one theme  ). Let me know if there are any bugs or missing things. Also feel free to modify it, it's for community after all.

How to install: unpack to ~/.atom/packages on Linux, other systems probably have similar path. Also, most of the files were generated automatically, I will probably delete some of them as they don't seem to be necessary.

EDIT: Oh, I forgot to mention that Atom will automatically detect the SatCom files if they begins with \b; line or have .satcom extension.

[krzys_h]

I believe you can create .css files in "styles" directory  and they will be loaded automatically (after restart). Be careful with colors through, as you noticed they may look bad with some styles.

EDIT: I just love this auto generated test :3

Code:

expect("life").toBe "easy"

[Simbax]

Here we go: https://github.com/MrSimbax/colobot-development-pack

Feel free to use and contribute. I'm wondering if we should publish it on atom.io.

Edit: ask for repository access if you want to modify it.

Edit2: In case someone didn't notice, the package is now available on atom.io. You can easily install it directly from GUI.