updated documentation

README.md

@@ -1,5 +1,7 @@
 # Welcome to ThreadR
-This is the source code for the ThreadR Forum Engine. It originated as a school project with the goal of developing a mix between a forum engine and a social media platform. When school was over, we left the project up for some time with the general intention to continue working on it until I took it down after an extended period of inactivity to host my own website on my server.
+This is the source code for the ThreadR Forum Engine.
+
+The project originated as a school project with the goal of developing a mix between a forum engine and a social media platform. When school was over, we left the project up for some time with the general intention to continue working on it until I took it down after an extended period of inactivity to host my own website on my server.
 
 Now, that it is being revived, the original scope of the project doesn’t really make sense anymore (at least to me) so it needs to shift slightly. Below is a list of goals that I would like to see achieved, feel free to discuss this in the issues or commit comments.
 
@@ -20,11 +22,6 @@ Once these two are given, here are some additional goals both from the original
 
 \- BodgeMaster
 
-# WARNING
-~~All information below is only relevant for documentation purposes.~~ This is a backup of the ThreadR project. A backup of the databases has been saved to `ThreadR.sql` and `web.sql`.
-
-Update: This project will (hopefully) be brought back to life in its now home - here. This warning will be removed once the new box is set up with ThreadR in its current state.
-
 # Git based automatic web deployment system
 This repository will be automagically pulled by the web server each time something is pushed by a user.
 
@@ -32,47 +29,34 @@ Dear Developers, Please use pushes sparingly because it takes a while for the se
 
 ## Symlinks
 The following files and directories are linked to areas where they can be accessed by the web server:
-  * `threadr/` → `threadr.lostcave.ddnss.de/threadr/` (all files acessible by the web server)
+  * `threadr/` → `threadr.lostcave.ddnss.de/threadr/` (all files acessible by the web server, READMEs get deleted on deployment)
   * `default.html` → `threadr.lostcave.ddnss.de/index.html` (http redirect)
 
-## Variables handled by the deployment script
-~~Here is the place for variables that will be replaced automatically on the server. Using them is encouraged. Using the paths is DISCOURAGED.
-This way, it is possible to change folder names around and fix the URLs in one place for all files simultaneously.~~
-
-This section has moved. Look in the files or their documentation below.
-
-## Some server variables that might turn out to be useful (example):
-```
-{
-  "HTTP_USER_AGENT":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:73.0) Gecko\/20100101 Firefox\/73.0",
-  "REMOTE_ADDR":"10.1.1.3"
-}
-```
-
 # Individual documentation for each file
 ### [[DIR] threadr](./threadr)
 This folder contains all the files that are parts of ThreadR directly
 ### [[DIR] config](./config)
 A place to store the configuation for a specific ThreadR instance (contains official instance config for now, will be moved elsewhere eventually)
 ### [[DIR] macros](./macros)
-files for variable_grabbler.py
+files for use with variable_grabbler.py
 ### [admin.php](./admin.php)
-This is the file that is shown on the internal admin page. It will contain a list of users, forums, threads, etc.
+~~This is the file that is shown on the internal admin page. It will contain a list of users, forums, threads, etc.
 At the moment, it is just a convenient way to access the other internal administration tools.
-This is not directly a part of ThreadR.
+This is not directly a part of ThreadR.~~
+
+This is part of the server management tools and therefore will be gone from this repository soon.
 ### [default.html](./default.html)
 The main index.html on the server. It redirects to ThreadR.
 ### [deployment_script.sh](./deployment_script.sh)
-This script is executed each time (well... if Gitea decides to actually run the WebHook) the repository gets pushed.
+This script is executed each time (or most of the time) the repository gets pushed.
 It contains the commands to execute the code variable replcement system and some other useful tasks.
-It’s working directory is the root of the git repository.
-If you want to know more: It is commented. Just look at it.
+Its working directory is the root of the git repository.
 ### [strassenkind.php](./strassenkind.php), [internal.css](./internal.css), [commands_status.conf](./commands_status.conf), [commands_ondemand.conf](./commands_ondemand.conf), [commands_git.conf](./commands_git.conf)
-The internal status page. strassenkind.php is the index, internal.css is the stylesheet strassenkind.php uses
+~~The internal status page. strassenkind.php is the index, internal.css is the stylesheet strassenkind.php uses
 and the .conf files are lists of commands to be executed by the status page. The status page auto-refreshes to display up-to-date information on the commands in commands_status.conf.
 It is advised to put commands that have a high disk usage or start/stop/restart/reload system services etc. on the commands_ondemand.conf file as the ondemand status page does not auto-refresh.
 The file commands_git.conf contains the commands that would be executed by the deployment WebHook. These are there to provide a fast and simple backup solution.
-The alternative to foce-updating ThreadR via the status page would be:
+The alternative to foce-updating ThreadR via the status page would be:~~
 ```
 ssh <user>@<threadr.ip|no public access set up currently>
 cd /var/www/git
@@ -84,14 +68,22 @@ cd web-deployment
 exit
 logout
 ```
-DEPRECATED: The administration framework will be turned into its own project.
+DEPRECATED: The administration framework will be turned into its own project and completely overhauled.
+### [LICENSE.md](./LICENSE.md)
+A copy of the Apache 2.0 license, the license this project is under
+### [NOTICE](./NOTICE)
+Copyright notice in plain text format
 ### [README.md](./README.md)
 this file
 ### [variable_grabbler.py](./variable_grabbler.py)
-This is the code variable replacer. It takes two arguments: The configuration file to be used and the file to be worked on.
-The configuration file contains a json which defines replacement string for each code variable in the format `"<VARNAME>":"<String>"`. Code variables are preceeded and followed by a % sign in code but this is not the case in the configuration.
-Also, they must be capitalized in code.
-An alternative option to providing a string in the config is to add arrays defining files or commands (Commands are not implemented yet.). Format: `"<VARNAME>":["<file|exec>","<filepath|command>"]
-Read the source code for further information.
+Custom macro processor, takes two arguments: macro declaration file and the file to be processed
 
-NOTICE: This file (or rather a more up-to-date version of it) will be moved to a new repository containing the deployment system.
+Macros in code are strings of capitalized characters prefixed and suffixed with %.
+
+Macro definition format: JSON
+"<MACRO>":"<text>" → direct replacement
+"<MACRO>":["file","<file path>"] → insert file
+"<MACRO>":["exec","<command>"] → run command and insert its output from stdout
+
+~~NOTICE: This file (or rather a more up-to-date version of it) will be moved to a new repository containing the deployment system.~~
+I haven’t exactly figured out how to handle this in the future. It is absolutely necessary to deploy a ThreadR instance because it is used to configure ThreadR so we need a copy of it here.

ThreadR.sql

@@ -1,144 +0,0 @@
--- phpMyAdmin SQL Dump
--- version 5.0.1
--- https://www.phpmyadmin.net/
---
--- Host: localhost
--- Generation Time: Jun 14, 2020 at 10:02 PM
--- Server version: 5.7.30-0ubuntu0.18.04.1
--- PHP Version: 7.2.24-0ubuntu0.18.04.6
-
-SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
-SET AUTOCOMMIT = 0;
-START TRANSACTION;
-SET time_zone = "+00:00";
-
-
-/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
-/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
-/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
-/*!40101 SET NAMES utf8mb4 */;
-
---
--- Database: `ThreadR`
---
-
--- --------------------------------------------------------
-
---
--- Table structure for table `boards`
---
-
-CREATE TABLE `boards` (
-  `board_id` bigint(20) NOT NULL,
-  `name_technical` varchar(100) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'url normally defined by name; switched over to name and id when duplicate found, should be base64, just in case',
-  `name_user-friendly` varchar(100) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'should be base64 encoded',
-  `description` varchar(4096) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL COMMENT 'should be base64 encoded',
-  `private` tinyint(1) NOT NULL DEFAULT '0',
-  `public_visible` tinyint(1) NOT NULL DEFAULT '0',
-  `time_created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
-  `time_altered` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-
--- --------------------------------------------------------
-
---
--- Table structure for table `posts`
---
-
-CREATE TABLE `posts` (
-  `post_id` bigint(48) NOT NULL,
-  `board_id` bigint(20) NOT NULL,
-  `user_id` int(11) NOT NULL,
-  `time_created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
-  `time_altered` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
-  `content` text,
-  `attachment_hash` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL COMMENT 'length determined by max file name size on ext4',
-  `attachment_file-name` varchar(255) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL COMMENT 'should be base64 encoded, length determined by max file name size on ext4',
-  `title` varchar(100) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'should be base64 encoded'
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-
--- --------------------------------------------------------
-
---
--- Table structure for table `profiles`
---
-
-CREATE TABLE `profiles` (
-  `user_id` int(11) NOT NULL,
-  `email` varchar(200) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'should be base64 encoded',
-  `avatar_hash` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL,
-  `real_name` varchar(100) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL COMMENT 'should be base64 encoded',
-  `about` varchar(8192) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL COMMENT 'the user''s bio (or whatever), should be base64 encoded',
-  `website` varchar(4096) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL COMMENT 'should be base64 encoded',
-  `status` varchar(512) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL COMMENT 'should be base64 encoded',
-  `time_altered` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-
--- --------------------------------------------------------
-
---
--- Table structure for table `users`
---
-
-CREATE TABLE `users` (
-  `user_id` int(11) NOT NULL COMMENT 'This is never to be given to the user. Session management will happen in-house.',
-  `name` varchar(40) CHARACTER SET ascii COLLATE ascii_bin NOT NULL COMMENT 'should be base64 encoded',
-  `time_created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
-  `time_altered` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'This is only for changing core user information. Used for checks like "Did X update their password?"',
-  `verified` tinyint(1) NOT NULL DEFAULT '0',
-  `authentication_algorithm` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
-  `authentication_salt` varchar(24) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
-  `authentication_string` varchar(224) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT 'This should be an impossible situation.' COMMENT 'Default value cause Yeeee!'
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-
---
--- Indexes for dumped tables
---
-
---
--- Indexes for table `boards`
---
-ALTER TABLE `boards`
-  ADD PRIMARY KEY (`board_id`);
-
---
--- Indexes for table `posts`
---
-ALTER TABLE `posts`
-  ADD PRIMARY KEY (`post_id`);
-
---
--- Indexes for table `profiles`
---
-ALTER TABLE `profiles`
-  ADD PRIMARY KEY (`user_id`),
-  ADD UNIQUE KEY `EMAIL` (`email`);
-
---
--- Indexes for table `users`
---
-ALTER TABLE `users`
-  ADD PRIMARY KEY (`user_id`),
-  ADD UNIQUE KEY `USERNAME` (`name`),
-  ADD UNIQUE KEY `id` (`user_id`);
-
---
--- AUTO_INCREMENT for dumped tables
---
-
---
--- AUTO_INCREMENT for table `boards`
---
-ALTER TABLE `boards`
-  MODIFY `board_id` bigint(20) NOT NULL AUTO_INCREMENT;
-
---
--- AUTO_INCREMENT for table `users`
---
-ALTER TABLE `users`
-  MODIFY `user_id` int(11) NOT NULL AUTO_INCREMENT COMMENT 'This is never to be given to the user. Session management will happen in-house.';
-COMMIT;
-
-/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
-/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
-/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;

config/README.md

@@ -1,16 +1,16 @@
 # Files
 ### [about.template](./about.template)
-The customizable part of the content page, will eventually be replaced with a markdown file
-### [names.conf](./names.conf)
-Information about the installation location of ThreadR
-- domain name
-- ThreadR directory on that domain
+The customizable part of the content page, HTML code for now, will eventually be replaced with a markdown file
+### [instance.conf](./instance.conf)
+configuration for a specific instance
+- domain_name=<public facing domain name of the instance>
+- threadr_dir=<directory on the webhost>
+- db_username=<mysql user>
+- db_password=<mysql password>
 
-Format: <config option>=<value>
+The format is strictly `<option>=<value>` because the mechanism used to load the config values is very simple. Don’t add additional whitespace for fancy formatting. Things *will* break. You have been warned.
 
-This format is strictly enforced because the mechanism to load the config is very primitive. Don’t add additional whitespace for fancy formatting. Things *will* break. You have been warned.
-
-How it is integrated: exec section of variable grabbler `/<config option>=/s/.*=//p" config/names.conf`
+The config is loaded once on deployment by the "variable grabbler" macro processor with `["exec","echo -n \"$(sed --quiet \"/<option>=/s/.*=//p\" config/instance.conf)\""]`.
 
 ### [README.md](./README.md)
 this file

macros/README.md

@@ -1,15 +1,27 @@
 # Files
-### [variable_grabbler.pass0.json](./variable_grabbler.pass0.json), [variable_grabbler.pass1.json](./variable_grabbler.pass1.json), [variable_grabbler.pass2.json](./variable_grabbler.pass2.json)
-These are the the config files for ../variable_grabbler.py.
-A short description of what the variables do and where they should go:
-  * `%NAVBAR%` (in HTML part of PHP file) → the Navbar, also includes an automatic logout-function if the user is in a place where they should never be logged in (triggered by absence of the $login variable)
-  * `%SET_LOGIN_VARIABLE%` (in PHP code) → sets the $login variable in PHP by detecting if the user has a valid logged in session
-  * `%NO_CHEAP_LOGIN_STEALING%` (in PHP code, before any non-header data is sent to client) → prevents the most primitive cookie stealing attempts, nothing advanced though
-  * `%BANNER_COOKIES%` (in HTML part of PHP file) → the cookie banner that every web page has
-  * `%PLEAZE_NO_CACHE%` (in PHP code, before any non-header data is sent to client) → requests the browser to not cache this page
-  * `%FORCE_LOGOUT%` (in PHP code) → logout immediately (obviously less common than the other variables...)
-  * `%CONTENT_DIR%` (anywhere in code) → the path to the ThreadR directory (without domain name)
-  * `%STYLESHEET%` (in HTML) → adds the tag linking the stylesheet
-  * `%REQUIRE_LOGIN%` (in PHP code, before any non-header data is sent to client) → sends users to the login page if they are not logged in
+### [[DIR] templates](./templates)
+contains the templates used by the macro processor
+### [pass0_templates.json](./pass0_templates.json)
+Macros that insert templates
+- `%NAVBAR%` the navigation bar at the top of the website (requires PHP)
+- `%BANNER_COOKIES%` the cookie banner (requires PHP)
+### [pass1_misc.json](./pass1_misc.json)
+Miscellaneous macros
+- `%STYLESHEET%` links stylesheet (HTML <head> section)
+- `%PLEAZE_NO_CACHE%` tell the browser not to cache this page (PHP, before any data is sent to the client)
+- `%COLOR_BEIGE%`, `%COLOR_ORANGE%`, `%COLOR_BLUE%`, `%COLOR_PINK%`, `%COLOR_CYAN%` color codes, format: #XXXXXX
+### [pass2_session.json](./pass2_session.json)
+Session handling macros
+- `%REQUIRE_LOGIN%` go to login page if not logged in
+- `%NO_CHEAP_SESSION_STEALING%` make sure that IP and user agent stay the same, otherwise end session
+- `%SET_LOGIN_VARIABLE%` sets the variable $login based on whether the user is logged in
+- `%FORCE_LOGOUT%` end session right here and now
+### [pass3_install-config.json](./pass3_install-config.json)
+Macros that grab the configuration values and inject them into the instance
+- `%DOMAIN_NAME%` the domain name of the instance
+- `%CONTENT_DIR%` the directory of the ThreadR home page on the webhost
+- `%DB_PASSWORD%` password for the MySQL server
+- `%DB_USERNAME%` username for the MySQL server
+- `%ABOUT_PAGE%` the template for the about page
 ### [README.md](./README.md)
 this file

macros/templates/README.md

@@ -3,3 +3,5 @@
 contains the navbar, depends on a variable in PHP to determine which field on the navbar should be highlighted
 ### [banner_cookies.template](./banner_cookies.template)
 contains the cookie banner
+### [README.md](./README.md)
+this file