Restructure documentation

Signed-off-by: Roberto Rosario <roberto.rosario@mayan-edms.com>

docs/_static/css/custom.css

@@ -3,3 +3,15 @@
     background: #2c3e50;
     padding: 3px;
 }
+
+.document a.internal {
+    text-decoration: none;
+    border-bottom: 1px dotted #707070;
+}
+
+.document a.internal:visited {
+    color: #2980B9;
+    text-decoration: none;
+    border-bottom: 1px dotted #707070;
+}
+

docs/chapters/acls.txt

@@ -1,6 +1,6 @@
-********************
+====================
 Access control lists
-********************
+====================
 
 Besides the permissions system explained in :doc:`../chapters/permissions`,
 Mayan EDMS provides per object permission granting. This feature is used to

docs/chapters/app_creation.rst

@@ -1,159 +0,0 @@
-************
-App creation
-************
-
-Mayan EDMS apps are essentially Django app with some extra code to register
-navigation, permissions and other relationships.
-
-
-App modules
-===========
-
-- __init__.py
-
-  Should be empty if possible. No initialization code should be here, use the
-  ready() method of the MayanAppConfig class in the apps.py module.
-
-- admin.py
-
-  Standard Django app module to define how models are to be presented in the
-  admin interface.
-
-- api_views.py
-
-  REST API views go here. Mayan EDMS uses Django REST Framework API view
-  classes.
-
-- apps.py
-
-  Contains the MayanAppConfig subclass as required by Django 1.7 and up. This
-  is a place to define the app name and translatable verbose name as well as
-  code to be execute when the modules of the app are ready.
-
-- classes.py
-
-  Hold python classes to be used internally or externally. Any class defined by
-  the app that is not a model.
-
-- events.py
-
-  Define event class instances that are later committed to a log by custom
-  code.
-
-- exceptions.py
-
-  Custom exceptions defined by the app.
-
-- fields.py
-
-  Place any custom form field classed you define here.
-
-- forms.py
-
-  Standard Django app module that hold custom form classes.
-
-- handlers.py
-
-  Contains the signal handlers, functions that will process a given signal
-  emitted from this or other apps. Connect the handler functions to the
-  corresponding signal in the ready() method of the MayanAppConfig subclass in
-  apps.py
-
-- html_widgets.py
-
-  Classes to render an HTML widget. HTML widget are not the same as Django's
-  native form widgets. Form widgets only work as part of a form field.
-  HTML widgets are for use outside of forms, such as in a table cell.
-
-- licenses.py
-
-  This module outlines the license text of the third party content used in 
-  the app. It could be other Python libraries, JavaScript libraries, etc.  
-
-- links.py
-
-  Defines the links to be used by the app. Import only from the navigation app
-  and the local permissions.py file.
-
-- literals.py
-
-  Stores magic numbers, module choices (if static), settings defaults, and
-  constants. Should contain all capital case variables. Must not import from
-  any other module.
-
-- managers.py
-
-  Standard Django app module that hold custom model managers. These act as
-  model class method to performs actions in a series of model instances or
-  utilitarian actions on external models instances.
-
-- models.py
-
-  Standard Django app module that defines ORM persistent data schema.
-
-- permissions.py
-
-  Defines the permissions to be used to validate user access by links and views.
-  Imports only from the permissions app. Link or view conditions such as
-  testing for is_staff or is_super_user flag are defined in this same module.
-
-- runtime.py
-
-  Use this module when you need the same instance of a class for the entire app.
-  This module acts as a shared memory space for the other modules of the app or
-  other apps.
-
-- search.py
-
-  Search model definitions. Define which field of the app's models are
-  searchable.
-
-- serializers.py
-
-  Hold Django REST Framework serializers used by the api_views.py module.
-
-- settings.py
-
-  Define the configuration settings instances that the app will use.
-
-- signals.py
-
-  Any custom defined signal goes here.
-
-- statistics.py
-
-  Provides functions that will compute any sort of statistical information on
-  the app’s data.
-
-- tasks.py
-
-  Code to be execute in the background or as an out-of-process action.
-
-- tests/ directory
-
-  Hold test modules. There should be one test_*.py module for each aspect being
-  tested, examples: test_api.py, test_views.py, test_parsers.py, test_permissions.py
-  Any shared constant data used by the tests should be added to tests/literals.py
-
-- utils.py
-
-  Holds utilitarian code that doesn't fit on any other app module or that is
-  used by several modules in the app. Anything used internally by the app that
-  is not a class or a literal (should be as little as possible)
-
-- widgets.py
-
-  Custom form widgets go here. This should be the only place with presentation
-  directives in the app (aside the templates).
-
-
-Views
-=====
-
-The module common.generics provides custom generic class based views to be used.
-The basic views used to create, edit, view and delete objects in Mayan EDMS
-are: SingleObjectCreateView, SingleObjectDetailView, SingleObjectEditView,
-and SingleObjectListView
-
-These views handle aspects relating to view permissions, object permissions,
-post action redirection and template context generation.

docs/chapters/backups.txt

@@ -1,6 +1,6 @@
-*******
+=======
 Backups
-*******
+=======
 
 To backup your install of Mayan EDMS just copy the actual document files and
 the database content. If you are using the default storage backend, the

docs/chapters/caa_entity.txt

@@ -1,6 +1,6 @@
-**************************************************
+==================================================
 Mayan EDMS Entity Contributor Assignment Agreement
-**************************************************
+==================================================
 
 Thank you for your interest in contributing to Mayan EDMS ("We" or "Us").
 
@@ -14,6 +14,7 @@ software project managed by Us.
 
 1. Definitions
 ==============
+
 "You" means any Legal Entity on behalf of whom a Contribution has been received
 by Us. "Legal Entity" means an entity which is not a natural person.
 "Affiliates" means other Legal Entities that control, are controlled by, or
@@ -55,6 +56,7 @@ first Submit a Contribution to Us, whichever is earlier.
 2. Grant of Rights
 ==================
 
+
 2.1 Copyright Assignment
 ------------------------
 
@@ -101,6 +103,7 @@ license the Contribution only under the terms of the license or licenses which
 We are using on the Submission Date for the Material (including any rights to
 adopt any future version of a license if permitted).
 
+
 2.4 Moral Rights
 ----------------
 
@@ -108,12 +111,14 @@ If moral rights apply to the Contribution, to the maximum extent permitted by
 law, You waive and agree not to assert such moral rights against Us or our
 successors in interest, or any of our licensees, either direct or indirect.
 
+
 2.5 Our Rights
 --------------
 
 You acknowledge that We are not obligated to use Your Contribution as part of
 the Material and may decide to include any Contribution We consider appropriate.
 
+
 2.6 Reservation of Rights
 -------------------------
 
@@ -159,6 +164,7 @@ OR OTHERWISE) UPON WHICH THE CLAIM IS BASED.
 6. Miscellaneous
 ================
 
+
 6.1 Jurisdiction
 ----------------
 
@@ -170,19 +176,25 @@ and the parties intend to avoid the application of the UN Convention to this
 Agreement and, thus, exclude the application of the UN Convention in its
 entirety to this Agreement.
 
+
 6.2 Acceptance
 --------------
+
 This Agreement sets out the entire agreement between You and Us for Your
 Contributions to Us and overrides all other agreements or understandings.
 
+
 6.3 Third parties
 -----------------
+
 If You or We assign the rights or obligations received through this Agreement
 to a third party, as a condition of the assignment, that third party must
 agree in writing to abide by all the rights and obligations in the Agreement.
 
+
 6.4 Unmet responsibilities
 --------------------------
+
 The failure of either party to require performance by the other party of any
 provision of this Agreement in one situation shall not affect the right of a
 party to require such performance at any time in the future. A waiver of
@@ -190,8 +202,10 @@ performance under a provision in one situation shall not be considered a
 waiver of the performance of the provision in the future or a waiver of the
 provision in its entirety.
 
+
 6.5 Continuation
 ----------------
+
 If any provision of this Agreement is found void and unenforceable, such
 provision will be replaced to the extent possible with a provision that comes
 closest to the meaning of the original provision and which is enforceable.

docs/chapters/caa_individual.txt

@@ -1,6 +1,6 @@
-******************************************************
+======================================================
 Mayan EDMS Individual Contributor Assignment Agreement
-******************************************************
+======================================================
 
 Thank you for your interest in contributing to Mayan EDMS ("We" or "Us").
 
@@ -11,6 +11,7 @@ caa@mayan-edms.com. This is a legally binding document, so please read it
 carefully before agreeing to it. The Agreement may cover more than one
 software project managed by Us.
 
+
 1. Definitions
 ==============
 
@@ -43,9 +44,11 @@ or otherwise designated in writing by You as "Not a Contribution."
 "Effective Date" means the date You execute this Agreement or the date You first
 Submit a Contribution to Us, whichever is earlier.
 
+
 2. Grant of Rights
 ==================
 
+
 2.1 Copyright Assignment
 ------------------------
 
@@ -87,6 +90,7 @@ and provided that this license is conditioned upon compliance with Section 2.3.
 
 2.3 Outbound License
 --------------------
+
 As a condition on the grant of rights in Sections 2.1 and 2.2, We agree to
 license the Contribution only under the terms of the license or licenses which
 We are using on the Submission Date for the Material (including any rights to
@@ -156,6 +160,7 @@ OR OTHERWISE) UPON WHICH THE CLAIM IS BASED.
 6. Miscellaneous
 ================
 
+
 6.1 Jurisdiction
 ----------------
 
@@ -167,12 +172,14 @@ and the parties intend to avoid the application of the UN Convention to this
 Agreement and, thus, exclude the application of the UN Convention in its
 entirety to this Agreement.
 
+
 6.2 Acceptance
 --------------
 
 This Agreement sets out the entire agreement between You and Us for Your
 Contributions to Us and overrides all other agreements or understandings.
 
+
 6.3 Third parties
 -----------------
 
@@ -180,6 +187,7 @@ If You or We assign the rights or obligations received through this Agreement
 to a third party, as a condition of the assignment, that third party must
 agree in writing to abide by all the rights and obligations in the Agreement.
 
+
 6.4 Unmet responsibilities
 --------------------------
 
@@ -190,6 +198,7 @@ performance under a provision in one situation shall not be considered a
 waiver of the performance of the provision in the future or a waiver of the
 provision in its entirety.
 
+
 6.5 Continuation
 ----------------
 

docs/chapters/cabinets.txt

@@ -1,13 +1,11 @@
+========
 Cabinets
 ========
 
 Cabinets are a multi-level method to organize documents. Each cabinet can
-contain documents as well as other sub level cabinets. Cabinets can
+contain documents as well as other sub level cabinets.
-be added from the :menuselection:`Cabinets --> Add new` menu.
 
 With a cabinet structure created, the next step is to add documents to them.
-To add documents to a cabinet, select the cabinet tab of a document, and
-select :menuselection:`Actions --> Add to cabinets` link.
 
 Documents can also be added in bulk to one or more cabinets from any view
 that lists documents.

docs/chapters/checkouts.txt

@@ -1,6 +1,6 @@
-*********
+=========
 Checkouts
-*********
+=========
 
 Checkouts are a way to block certain accesses or actions of a document for a
 period of time.

docs/chapters/code_structure.txt

@@ -0,0 +1,242 @@
+==============
+Code structure
+==============
+
+
+Follow PEP8
+-----------
+
+Whenever possible, but don't obsess over things like line length:
+
+.. code-block:: bash
+
+    $ flake8 --ignore=E501,E128,E122 |less
+
+To perform automatic PEP8 checks, install flake8's git hook using:
+
+.. code-block:: bash
+
+    $ flake8 --install-hook git
+
+
+Imports
+-------
+
+Import order should be:
+
+- Standard Python modules
+- Installed Python modules
+- Core Django modules
+- Installed Django modules
+- Mayan EDMS modules
+- Local imports
+
+Example:
+
+.. code-block:: bash
+
+    from __future__ import absolute_import
+
+    # Standard Python library
+    import base64
+
+    # 3rd party installed Python libraries
+    import requests
+
+    # Django core modules
+    from django.db.models import Q
+    from django.template.defaultfilters import slugify
+    from django.utils.translation import ugettext
+    from django.utils.translation import ugettext_lazy as _
+
+    # 3rd party installed Django libraries
+    from rest_framework import APIView
+
+    # Mayan apps
+    from metadata.classes import MetadataClass
+
+    # Local app imports (relative)
+    from .conf.settings import (
+        AVAILABLE_INDEXING_FUNCTIONS,
+        MAX_SUFFIX_COUNT, SLUGIFY_PATHS
+    )
+    from .exceptions import MaxSuffixCountReached
+    from .filesystem import (
+        fs_create_index_directory, fs_create_document_link,
+        fs_delete_document_link, fs_delete_index_directory,
+        assemble_suffixed_filename
+    )
+    from .models import Index, IndexInstanceNode, DocumentRenameCount
+
+All local app module imports are in relative form. Local app module name is to
+be referenced as little as possible, unless required by a specific feature,
+trick, restriction (e.g., Runtime modification of the module's attributes).
+
+Incorrect:
+
+.. code-block:: bash
+
+
+    # documents app views.py model
+    from documents.models import Document
+
+Correct:
+
+.. code-block:: bash
+
+    # documents app views.py model
+    from .models import Document
+
+
+Dependencies
+------------
+
+Mayan EDMS apps follow a hierarchical model of dependency. Apps import from
+their parents or siblings, never from their children. Think plugins. A parent
+app must never assume anything about a possible existing child app. The
+documents app and the Document model are the basic entities; they must never
+import anything else. The common and main apps are the base apps.
+
+
+Variables
+---------
+
+Naming of variables should follow a Major to Minor convention, usually
+including the purpose of the variable as the first piece of the name, using
+underscores as spaces. camelCase is not used in Mayan EDMS.
+
+Examples:
+
+Links:
+
+.. code-block:: bash
+
+    link_document_page_transformation_list = ...
+    link_document_page_transformation_create = ...
+    link_document_page_transformation_edit = ...
+    link_document_page_transformation_delete = ...
+
+Constants:
+
+.. code-block:: bash
+
+    PERMISSION_SMART_LINK_VIEW = ...
+    PERMISSION_SMART_LINK_CREATE = ...
+    PERMISSION_SMART_LINK_DELETE = ...
+    PERMISSION_SMART_LINK_EDIT = ...
+
+Classes:
+
+.. code-block:: bash
+
+    class Document(models.Model):
+    class DocumentPage(models.Model):
+    class DocumentPageTransformation(models.Model):
+    class DocumentType(models.Model):
+    class DocumentTypeFilename(models.Model):
+
+
+Strings
+-------
+
+Quotation character used in Mayan EDMS for strings is the single quote.
+Double quote is used for multiple line comments or HTML markup.
+
+
+Migrations
+----------
+
+Migrations should do only one thing (example: either create a table, move data
+to a new table or remove an old table) to aid retrying on failure.
+
+
+General
+-------
+
+Code should appear in their modules in alphabetic order or in their order of
+importance if it makes more sense for the specific application. This makes
+visual scanning easier on modules with a large number of imports, views or
+classes. Class methods that return a value should be pretended with a
+``get_`` to differentiate from an object’s properties. When a variable refers
+to a file it should be named as follows:
+
+- filename:  The file’s name and extension only.
+- filepath:  The entire path to the file including the filename.
+- path:  A path to a directory.
+
+Flash messages should end with a period as applicable for the language.
+Only exception is when the tail of the message contains an exceptions message
+as passed directly from the exception object.
+
+
+Steps to deploy a development version
+=====================================
+
+.. code-block:: bash
+
+    $ git clone https://gitlab.com/mayan-edms/mayan-edms.git
+    $ cd mayan-edms
+    $ git checkout <Corresponding branch>
+    $ virtualenv venv
+    $ source venv/bin/activate
+    $ pip install -r requirements.txt
+    $ ./manage.py initialsetup
+    $ ./manage.py runserver
+
+
+
+Debugging
+=========
+
+Mayan EDMS makes extensive use of Django's new
+:django-docs:`logging capabilities <topics/logging>`.
+
+By default debug logging for all apps is turned on. If you wish to customize
+how logging is managed turn off automatic logging by setting
+`COMMON_AUTO_LOGGING` to ``False`` and add the following lines to your
+``settings/local.py`` file::
+
+    LOGGING = {
+        'version': 1,
+        'disable_existing_loggers': True,
+        'formatters': {
+            'verbose': {
+                'format': '%(levelname)s %(asctime)s %(name)s %(process)d %(thread)d %(message)s'
+            },
+            'intermediate': {
+                'format': '%(name)s <%(process)d> [%(levelname)s] "%(funcName)s() %(message)s"'
+            },
+            'simple': {
+                'format': '%(levelname)s %(message)s'
+            },
+        },
+        'handlers': {
+            'console':{
+                'level':'DEBUG',
+                'class':'logging.StreamHandler',
+                'formatter': 'intermediate'
+            }
+        },
+        'loggers': {
+            'documents': {
+                'handlers':['console'],
+                'propagate': True,
+                'level':'DEBUG',
+            },
+            'common': {
+                'handlers':['console'],
+                'propagate': True,
+                'level':'DEBUG',
+            },
+        }
+    }
+
+
+Likewise, to see the debug output of the ``tags`` app, just add the following inside the ``loggers`` block::
+
+
+    'tags': {
+        'handlers':['console'],
+        'propagate': True,
+        'level':'DEBUG',
+    },

docs/chapters/contributing.txt

@@ -0,0 +1,18 @@
+============
+Contributing
+============
+
+Mayan EDMS is under active development, and contributions are welcome.
+
+If you have a feature request, suggestion or bug report, please open a new
+issue on the `GitLab issue tracker`_. To submit patches, please send a merge
+request on GitLab_.
+
+.. _GitLab: https://gitlab.com/mayan-edms/mayan-edms/
+.. _`GitLab issue tracker`: https://gitlab.com/mayan-edms/mayan-edms/issues
+
+
+Contributing changes
+====================
+Follow the latest contributing guidelines outlined here:
+https://gitlab.com/mayan-edms/mayan-edms/blob/master/CONTRIBUTING.md

docs/chapters/deploying.txt

@@ -1,6 +1,8 @@
-******************
+.. _deployment_basic:
-Direct deployments
+
-******************
+=================
+Direct deployment
+=================
 
 Mayan EDMS should be deployed like any other Django_ project and
 preferably using virtualenv_. Below are some ways to deploy and use Mayan EDMS.

docs/chapters/docker.rst

@@ -1,447 +0,0 @@
-============
-Docker image
-============
-
-How to use this image
-=====================
-
-.. _docker_install:
-
-Start a Mayan EDMS Docker image
--------------------------------
-
-1. Download the Mayan EDMS Docker image
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-With Docker properly installed, proceed to download the Mayan EDMS Docker
-image using the command::
-
-    docker pull mayanedms/mayanedms:|DOCKER_MAYAN_IMAGE_VERSION|
-
-Instead of a specific version tag you may use then generic ``latest`` tag
-to the get latest version available automatically. If you use the ``latest``
-tag here, remember to do so in the next steps also.::
-
-    docker pull mayanedms/mayanedms:latest
-
-
-2. Download the PostgreSQL Docker image
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: bash
-
-    docker pull |DOCKER_POSTGRES_IMAGE_VERSION|
-
-
-3. Download the Redis Docker image
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: bash
-
-    docker pull |DOCKER_REDIS_IMAGE_VERSION|
-
-4. Create and run a PostgreSQL container
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: bash
-
-    docker run \
-    -d \
-    --name mayan-edms-postgres \
-    --restart=always \
-    -p 5432:5432 \
-    -e POSTGRES_USER=mayan \
-    -e POSTGRES_DB=mayan \
-    -e POSTGRES_PASSWORD=mayanuserpass \
-    -v /docker-volumes/mayan-edms/postgres:/var/lib/postgresql/data \
-    |DOCKER_POSTGRES_IMAGE_VERSION|
-
-The PostgreSQL container will have one database named ``mayan``, with an user
-named ``mayan`` too, with a password of ``mayanuserpass``. The container will
-expose its internal 5432 port (PostgreSQL's default port) via the host's
-5432 port. The data of this container will reside on the host's
-``/docker-volumes/mayan-edms/postgres`` folder.
-
-
-5. Create and run a Redis container
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: bash
-
-    docker run \
-    -d \
-    --name mayan-edms-redis \
-    --restart=always \
-    -p 6379:6379
-    |DOCKER_REDIS_IMAGE_VERSION| \
-    redis-server \
-    --databases \
-    "2" \
-    --maxmemory-policy \
-    allkeys-lru \
-    --save \
-    ""
-
-The Redis container will have two databases, one for background task messages,
-and the other to hold the results of those background tasks. Redis is
-configure to not save its content to disk and to automatically clear up
-memory.
-
-6. Create and run a Mayan EDMS container
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. code-block:: bash
-
-
-    docker run \
-    -d \
-    --name mayan-edms \
-    --restart=always \
-    -p 80:8000 \
-    -e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'172.17.0.1'}}" \
-    -e MAYAN_CELERY_BROKER_URL="redis://172.17.0.1:6379/0" \
-    -e MAYAN_CELERY_RESULT_BACKEND="redis://172.17.0.1:6379/1" \
-    -v /docker-volumes/mayan-edms/media:/var/lib/mayan \
-    mayanedms/mayanedms:<version>
-
-The Mayan EDMS container will connect to the PostgreSQL container via the
-``172.17.0.1`` IP address (the Docker host's default IP address). It will
-connect using the ``django.db.backends.postgresql`` database driver and
-connect to the ``mayan`` database using the ``mayan`` user with the password
-``mayanuserpass``. The files of the container will be store in the
-host's ``/docker-volumes/mayan-edms/media`` folder. The container will
-expose its web service running on port 8000 on the host's port 80.
-
-The container will be available by browsing to ``http://localhost`` or to
-the IP address of the computer running the container.
-
-If another web server is running on port 80 use a different port in the
-``-p`` option. For example: ``-p 81:8000``.
-
-
-Using a dedicated Docker network
---------------------------------
-
-Use this method to avoid having to expose PostreSQL port to the host's network
-or if you have other PostgreSQL instances but still want to use the default
-port of 5432 for this installation.
-
-Create the network::
-
-    docker network create mayan
-
-Launch the PostgreSQL container with the network option and remove the port
-binding (``-p 5432:5432``)::
-
-    docker run \
-    -d \
-    --name mayan-edms-postgres \
-    --network=mayan \
-    --restart=always \
-    -e POSTGRES_USER=mayan \
-    -e POSTGRES_DB=mayan \
-    -e POSTGRES_PASSWORD=mayanuserpass \
-    -v /docker-volumes/mayan-edms/postgres:/var/lib/postgresql/data \
-    |DOCKER_POSTGRES_IMAGE_VERSION|
-
-Launch the Redis container with the network option and remove the port
-binding (``-p 6379:6379``)::
-
-    docker run \
-    -d \
-    --name mayan-edms-redis \
-    --network=mayan \
-    |DOCKER_REDIS_IMAGE_VERSION| \
-    redis-server \
-    --databases \
-    "2" \
-    --maxmemory-policy \
-    allkeys-lru \
-    --save \
-    ""
-
-Launch the Mayan EDMS container with the network option and change the
-database hostname to the PostgreSQL container name (``mayan-edms-postgres``)
-instead of the IP address of the Docker host (``172.17.0.1``)::
-
-    docker run \
-    -d \
-    --name mayan-edms \
-    --network=mayan \
-    --restart=always \
-    -p 80:8000 \
-    -e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'mayan-edms-postgres'}}" \
-    -e MAYAN_CELERY_BROKER_URL="redis://mayan-edms-redis:6379/0" \
-    -e MAYAN_CELERY_RESULT_BACKEND="redis://mayan-edms-redis:6379/1" \
-    -v /docker-volumes/mayan-edms/media:/var/lib/mayan \
-    mayanedms/mayanedms:<version>
-
-
-Stopping and starting the container
------------------------------------
-
-To stop the container use::
-
-    docker stop mayan-edms
-
-
-To start the container again::
-
-    docker start mayan-edms
-
-
-.. _docker_environment_variables:
-
-
-Environment Variables
----------------------
-
-The common set of settings can also be modified via environment variables when
-using the Docker image. In addition to the common set of settings, some Docker
-image specific environment variables are available.
-
-``MAYAN_SETTINGS_MODULE``
-
-Optional. Allows loading an alternate settings file.
-
-
-``MAYAN_GUNICORN_TIMEOUT``
-
-Optional. Changes the amount of time the frontend worker will wait for a
-request to finish before raising a timeout error. The default is 120
-seconds.
-
-``MAYAN_GUNICORN_WORKERS``
-
-Optional. This environment variable controls the number of frontend workers
-that will be executed. If not specified the default is 2. For heavier loads,
-user a higher number. A formula recommended for this setting is the number
-of CPU cores + 1.
-
-``MAYAN_WORKER_FAST_CONCURRENCY``
-
-Optional. Changes the concurrency (number of child processes) of the Celery
-worker consuming the queues in the fast (low latency, short tasks) category.
-Default is 1. Use 0 to disable hardcoded concurrency and allow the Celery
-worker to launch its default number of child processes (equal to the number
-of CPUs detected).
-
-``MAYAN_WORKER_MEDIUM_CONCURRENCY``
-
-Optional. Changes the concurrency (number of child processes) of the Celery
-worker consuming the queues in the medium (medium latency, long running tasks)
-category. Default is 1. Use 0 to disable hardcoded concurrency and allow the
-Celery worker to launch its default number of child processes (equal to the
-number of CPUs detected).
-
-``MAYAN_WORKER_SLOW_CONCURRENCY``
-
-Optional. Changes the concurrency (number of child processes) of the Celery
-worker consuming the queues in the slow (high latency, very long running tasks)
-category. Default is 1. Use 0 to disable hardcoded concurrency and allow the
-Celery worker to launch its default number of child processes (equal to the
-number of CPUs detected).
-
-``MAYAN_USER_UID``
-
-Optional. Changes the UID of the ``mayan`` user internal to the Docker
-container. Defaults to 1000.
-
-``MAYAN_USER_GID``
-
-Optional. Changes the GID of the ``mayan`` user internal to the Docker
-container. Defaults to 1000.
-
-
-Included drivers
-----------------
-
-The Docker image supports using Redis and RabbitMQ as result backends. For
-databases, the image includes support for PostgreSQL and MySQL/MariaDB.
-Support for additional brokers or databases may be added using the
-``MAYAN_APT_INSTALL`` environment variable.
-
-
-.. _docker-accessing-outside-data:
-
-Accessing outside data
-======================
-
-To use Mayan EDMS's staging folders or watch folders from Docker, the data
-for these source must be made accessible to the container. This is done by
-mounting the folders in the host computer to folders inside the container.
-This is necessary because Docker containers do not have access to host data
-on purpose. For example, to make a folder in the host accessible as a watch
-folder, add the following to the Docker command line when starting the
-container::
-
-    -v /opt/scanned_files:/scanned_files
-
-The command line would look like this::
-
-    docker run ... -v /opt/scanned_files:/scanned_files mayanedms/mayanedms:latest
-
-Now create a watch folder in Mayan EDMS using the path ``/scanned_files``
-and the documents from the host folder ``/opt/scanned_files`` will be
-automatically available. Use the same procedure to mount host folders to be
-used as staging folders. In this example ``/scanned_files`` was used as the
-container directory, but any path can be used as long as:
-
-- the path not an already existing path
-- the path is not used by any other program
-- the path is a single level path
-
-
-Performing backups
-==================
-
-To backup the existing data, stop the image and copy the content of the volume.
-For the example::
-
-    docker run -d --name mayan-edms --restart=always -p 80:8000 \
-    -v /docker-volumes/mayan:/var/lib/mayan \
-    -v /opt/scanned_files:/scanned_files mayanedms/mayanedms:latest
-
-That would be the ``/docker-volumes/mayan folder``::
-
-    sudo tar -zcvf backup.tar.gz /docker-volumes/mayan
-    sudo chown `whoami` backup.tar.gz
-
-If using an external PostgreSQL or MySQL database or database containers, these
-too need to be backed up using their respective procedures. A simple solution
-is to copy the entire database container volume after the container has
-been stopped.
-
-
-Restoring from a backup
-=======================
-
-Uncompress the backup archive in the original docker volume using::
-
-    sudo tar -xvzf backup.tar.gz -C /
-
-
-Upgrading
-=========
-
-Upgrading a Mayan EDMS Docker container is actually a matter of stopping and
-deleting the container, downloading the most recent version of the image and
-starting a container again. The container will take care of updating the
-database structure to the newest version if necessary.
-
-**IMPORTANT!** Do not delete the volume storing the data, only the container.
-
-Stop the container to be upgraded::
-
-    docker stop mayan-edms
-
-
-Remove the container::
-
-    docker rm mayan-edms
-
-
-Pull the new image version::
-
-    docker pull mayanedms/mayanedms:latest
-
-
-Start the container again with the new image version::
-
-    docker run -d --name mayan-edms --restart=always -p 80:8000 -v /docker-volumes/mayan:/var/lib/mayan mayanedms/mayanedms:latest
-
-
-Building the image
-==================
-
-Clone the repository with::
-
-    git clone https://gitlab.com/mayan-edms/mayan-edms.git
-
-Change to the directory of the cloned repository::
-
-    cd mayan-edms
-
-Execute Docker's build command using the provided makefile::
-
-    make docker-build
-
-Or using an APT cache to speed up the build::
-
-    make docker-build-with-proxy APT_PROXY=172.17.0.1:3142
-
-Replace the IP address `172.17.0.1` with the IP address of the computer
-running the APT proxy and caching service.
-
-
-Customizing the image
-=====================
-
-Simple method
--------------
-
-If you just need to add a few Ubuntu or Python packages to your installation,
-you can use the following environment variables:
-
-``MAYAN_APT_INSTALLS``
-
-Specifies a list of Ubuntu .deb packages to be installed via APT when the
-container is first created. The installed packages are not lost when the image
-is stopped. Example: To install the Tesseract OCR language packs for German
-and Spanish add the following in your ``docker start`` command line::
-
-    -e MAYAN_APT_INSTALLS="tesseract-ocr-deu tesseract-ocr-spa"
-
-``MAYAN_PIP_INSTALLS``
-
-Specifies a list of Python packages to be installed via ``pip``. Packages will
-be downloaded from the Python Package Index (https://pypi.python.org) by
-default.
-
-
-Using Docker compose
-====================
-
-To deploy a complete production stack using the included Docker compose file
-execute::
-
-    docker-compose -f docker-compose.yml up -d
-
-This Docker compose file will provision four containers:
-
-- Postgres as the database
-- Redis as the Celery result storage
-- RabbitMQ as the Celery broker
-- Mayan EDMS using the above service containers
-
-To stop the stack use::
-
-    docker-compose -f docker-compose.yml stop
-
-The stack will also create four volumes to store the data of each container.
-These are:
-
-- mayan_app - The Mayan EDMS data container, normally called `mayan_data` when not using Docker compose.
-- mayan_broker - The broker volume, in this case RabbitMQ.
-- mayan_db - The database volume, in this case Postgres.
-- mayan_results - The celery result backend volume, in this case Redis.
-
-
-Nightly images
-==============
-
-The continuous integration pipeline used for testing development builds also
-produces a resulting Docker image. These are build automatically and their
-stability is not guaranteed. They should never be used in production.
-If you want to try out the Docker images the development uses or want a sneak
-peek at the new features being worked on checkout the container registry at:
-https://gitlab.com/mayan-edms/mayan-edms/container_registry
-
-
-Executing commands on a container
-=================================
-
-To perform Mayan EDMS commands in a running container that will take all the
-settings from the container, use the following command::
-
-    docker exec -ti <container name> /usr/local/bin/entrypoint.sh "/opt/mayan-edms/bin/mayan-edms.py <command>"

docs/chapters/docker/accessing_data.txt

@@ -0,0 +1,30 @@
+.. _docker-accessing-outside-data:
+
+
+======================
+Accessing outside data
+======================
+
+To use Mayan EDMS's staging folders or watch folders from Docker, the data
+for these source must be made accessible to the container. This is done by
+mounting the folders in the host computer to folders inside the container.
+This is necessary because Docker containers do not have access to host data
+on purpose. For example, to make a folder in the host accessible as a watch
+folder, add the following to the Docker command line when starting the
+container::
+
+    -v /opt/scanned_files:/scanned_files
+
+The command line would look like this::
+
+    docker run ... -v /opt/scanned_files:/scanned_files mayanedms/mayanedms:latest
+
+Now create a watch folder in Mayan EDMS using the path ``/scanned_files``
+and the documents from the host folder ``/opt/scanned_files`` will be
+automatically available. Use the same procedure to mount host folders to be
+used as staging folders. In this example ``/scanned_files`` was used as the
+container directory, but any path can be used as long as:
+
+- the path not an already existing path
+- the path is not used by any other program
+- the path is a single level path

docs/chapters/docker/backups.txt

@@ -0,0 +1,32 @@
+.. _docker_backups:
+
+==================
+Performing backups
+==================
+
+To backup the existing data, stop the image and copy the content of the volume.
+For the example::
+
+    docker run -d --name mayan-edms --restart=always -p 80:8000 \
+    -v /docker-volumes/mayan:/var/lib/mayan \
+    -v /opt/scanned_files:/scanned_files mayanedms/mayanedms:latest
+
+That would be the ``/docker-volumes/mayan folder``::
+
+    sudo tar -zcvf backup.tar.gz /docker-volumes/mayan
+    sudo chown `whoami` backup.tar.gz
+
+If using an external PostgreSQL or MySQL database or database containers, these
+too need to be backed up using their respective procedures. A simple solution
+is to copy the entire database container volume after the container has
+been stopped.
+
+
+=======================
+Restoring from a backup
+=======================
+
+Uncompress the backup archive in the original docker volume using::
+
+    sudo tar -xvzf backup.tar.gz -C /
+

docs/chapters/docker/building.txt

@@ -0,0 +1,25 @@
+.. _docker_building:
+
+==================
+Building the image
+==================
+
+Clone the repository with::
+
+    git clone https://gitlab.com/mayan-edms/mayan-edms.git
+
+Change to the directory of the cloned repository::
+
+    cd mayan-edms
+
+Execute Docker's build command using the provided makefile::
+
+    make docker-build
+
+Or using an APT cache to speed up the build::
+
+    make docker-build-with-proxy APT_PROXY=172.17.0.1:3142
+
+Replace the IP address `172.17.0.1` with the IP address of the computer
+running the APT proxy and caching service.
+

docs/chapters/docker/commands.txt

@@ -0,0 +1,10 @@
+.. _docker_commands:
+
+=================================
+Executing commands on a container
+=================================
+
+To perform Mayan EDMS commands in a running container that will take all the
+settings from the container, use the following command::
+
+    docker exec -ti <container name> /usr/local/bin/entrypoint.sh "/opt/mayan-edms/bin/mayan-edms.py <command>"

docs/chapters/docker/customizing.txt

@@ -0,0 +1,23 @@
+.. _docker_install_networked:
+
+=====================
+Customizing the image
+=====================
+
+If you just need to add a few Ubuntu or Python packages to your installation,
+you can use the following environment variables:
+
+``MAYAN_APT_INSTALLS``
+
+Specifies a list of Ubuntu .deb packages to be installed via APT when the
+container is first created. The installed packages are not lost when the image
+is stopped. Example: To install the Tesseract OCR language packs for German
+and Spanish add the following in your ``docker start`` command line::
+
+    -e MAYAN_APT_INSTALLS="tesseract-ocr-deu tesseract-ocr-spa"
+
+``MAYAN_PIP_INSTALLS``
+
+Specifies a list of Python packages to be installed via ``pip``. Packages will
+be downloaded from the Python Package Index (https://pypi.python.org) by
+default.

docs/chapters/docker/docker_compose.txt

@@ -0,0 +1,30 @@
+.. _docker_compose:
+
+====================
+Using Docker compose
+====================
+
+To deploy a complete production stack using the included Docker compose file
+execute::
+
+    docker-compose -f docker-compose.yml up -d
+
+This Docker compose file will provision four containers:
+
+- Postgres as the database
+- Redis as the Celery result storage
+- RabbitMQ as the Celery broker
+- Mayan EDMS using the above service containers
+
+To stop the stack use::
+
+    docker-compose -f docker-compose.yml stop
+
+The stack will also create four volumes to store the data of each container.
+These are:
+
+- mayan_app - The Mayan EDMS data container, normally called `mayan_data` when not using Docker compose.
+- mayan_broker - The broker volume, in this case RabbitMQ.
+- mayan_db - The database volume, in this case Postgres.
+- mayan_results - The celery result backend volume, in this case Redis.
+

docs/chapters/docker/drivers.txt

@@ -0,0 +1,12 @@
+.. _docker_image_drivers:
+
+================
+Included drivers
+================
+
+The Docker image supports using Redis and RabbitMQ as result backends. For
+databases, the image includes support for PostgreSQL and MySQL/MariaDB.
+Support for additional brokers or databases may be added using the
+``MAYAN_APT_INSTALL`` environment variable.
+
+

docs/chapters/docker/environment_variables.txt

@@ -0,0 +1,63 @@
+.. index:: Docker environment variables
+.. _docker_environment_variables:
+
+
+=====================
+Environment Variables
+=====================
+
+The common set of settings can also be modified via environment variables when
+using the Docker image. In addition to the common set of settings, some Docker
+image specific environment variables are available.
+
+``MAYAN_SETTINGS_MODULE``
+
+Optional. Allows loading an alternate settings file.
+
+
+``MAYAN_GUNICORN_TIMEOUT``
+
+Optional. Changes the amount of time the frontend worker will wait for a
+request to finish before raising a timeout error. The default is 120
+seconds.
+
+``MAYAN_GUNICORN_WORKERS``
+
+Optional. This environment variable controls the number of frontend workers
+that will be executed. If not specified the default is 2. For heavier loads,
+user a higher number. A formula recommended for this setting is the number
+of CPU cores + 1.
+
+``MAYAN_WORKER_FAST_CONCURRENCY``
+
+Optional. Changes the concurrency (number of child processes) of the Celery
+worker consuming the queues in the fast (low latency, short tasks) category.
+Default is 1. Use 0 to disable hardcoded concurrency and allow the Celery
+worker to launch its default number of child processes (equal to the number
+of CPUs detected).
+
+``MAYAN_WORKER_MEDIUM_CONCURRENCY``
+
+Optional. Changes the concurrency (number of child processes) of the Celery
+worker consuming the queues in the medium (medium latency, long running tasks)
+category. Default is 1. Use 0 to disable hardcoded concurrency and allow the
+Celery worker to launch its default number of child processes (equal to the
+number of CPUs detected).
+
+``MAYAN_WORKER_SLOW_CONCURRENCY``
+
+Optional. Changes the concurrency (number of child processes) of the Celery
+worker consuming the queues in the slow (high latency, very long running tasks)
+category. Default is 1. Use 0 to disable hardcoded concurrency and allow the
+Celery worker to launch its default number of child processes (equal to the
+number of CPUs detected).
+
+``MAYAN_USER_UID``
+
+Optional. Changes the UID of the ``mayan`` user internal to the Docker
+container. Defaults to 1000.
+
+``MAYAN_USER_GID``
+
+Optional. Changes the GID of the ``mayan`` user internal to the Docker
+container. Defaults to 1000.

docs/chapters/docker/index.txt

@@ -0,0 +1,34 @@
+.. index:: Docker
+
+======
+Docker
+======
+
+- Installation: :doc:`install_simple` | :doc:`install_docker_network` | :doc:`docker_compose` | :doc:`start_stop`
+- Setup: :doc:`environment_variables` | :doc:`drivers`
+- Integration: :doc:`accessing_data`
+- Development: :doc:`building` | :doc:`customizing` | :doc:`nightly`
+- Administration: :doc:`backups` | :doc:`upgrading` | :doc:`commands`
+
+
+Docker is a container technology. Containers are a standard unit of software
+that packages up code and all its dependencies.
+
+.. toctree::
+    :hidden:
+
+    install_simple
+    install_docker_network
+    start_stop
+    environment_variables
+    drivers
+    accessing_data
+    backups
+    upgrading
+    building
+    customizing
+    docker_compose
+    nightly
+    commands
+
+

docs/chapters/docker/install_docker_network.txt

@@ -0,0 +1,59 @@
+.. _docker_install_networked:
+
+================================
+Using a dedicated Docker network
+================================
+
+Use this method to avoid having to expose PostreSQL port to the host's network
+or if you have other PostgreSQL instances but still want to use the default
+port of 5432 for this installation.
+
+#. Create the network::
+
+       docker network create mayan
+
+#. Launch the PostgreSQL container with the network option and remove the
+   port binding (``-p 5432:5432``)::
+
+       docker run \
+       -d \
+       --name mayan-edms-postgres \
+       --network=mayan \
+       --restart=always \
+       -e POSTGRES_USER=mayan \
+       -e POSTGRES_DB=mayan \
+       -e POSTGRES_PASSWORD=mayanuserpass \
+       -v /docker-volumes/mayan-edms/postgres:/var/lib/postgresql/data \
+       |DOCKER_POSTGRES_IMAGE_VERSION|
+
+#. Launch the Redis container with the network option and remove the port
+   binding (``-p 6379:6379``)::
+
+       docker run \
+       -d \
+       --name mayan-edms-redis \
+       --network=mayan \
+       |DOCKER_REDIS_IMAGE_VERSION| \
+       redis-server \
+       --databases \
+       "2" \
+       --maxmemory-policy \
+       allkeys-lru \
+       --save \
+       ""
+
+#. Launch the Mayan EDMS container with the network option and change the
+   database hostname to the PostgreSQL container name (``mayan-edms-postgres``)
+   instead of the IP address of the Docker host (``172.17.0.1``)::
+
+       docker run \
+       -d \
+       --name mayan-edms \
+       --network=mayan \
+       --restart=always \
+       -p 80:8000 \
+       -e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'mayan-edms-postgres'}}" \
+       -e MAYAN_CELERY_BROKER_URL="redis://mayan-edms-redis:6379/0" \
+       -e MAYAN_CELERY_RESULT_BACKEND="redis://mayan-edms-redis:6379/1" \
+       -v /docker-volumes/mayan-edms/media:/var/lib/mayan \
+       mayanedms/mayanedms:<version>

docs/chapters/docker/install_simple.txt

@@ -0,0 +1,121 @@
+.. _docker_install:
+
+
+==========================
+Simple Docker installation
+==========================
+
+#. Install Docker
+
+   .. code-block:: bash
+
+       wget -qO- https://get.docker.com/ | sh
+
+   If you don't want run an automated script, follow the instructions outlined
+   in their documentation: https://docs.docker.com/install/
+
+   Once the Docker installation is finished, proceed to the link below to install
+   the Docker image for Mayan EDMS.
+
+
+#. Download the Mayan EDMS Docker image
+
+   With Docker properly installed, proceed to download the Mayan EDMS Docker
+   image using the command::
+
+       docker pull mayanedms/mayanedms:|DOCKER_MAYAN_IMAGE_VERSION|
+
+   Instead of a specific version tag you may use then generic ``latest`` tag
+   to the get latest version available automatically. If you use the ``latest``
+   tag here, remember to do so in the next steps also.::
+
+       docker pull mayanedms/mayanedms:latest
+
+
+#. Download the PostgreSQL Docker image
+
+   .. code-block:: bash
+
+       docker pull |DOCKER_POSTGRES_IMAGE_VERSION|
+
+
+#. Download the Redis Docker image
+
+   .. code-block:: bash
+
+       docker pull |DOCKER_REDIS_IMAGE_VERSION|
+
+
+#. Create and run a PostgreSQL container
+
+   .. code-block:: bash
+
+       docker run \
+       -d \
+       --name mayan-edms-postgres \
+       --restart=always \
+       -p 5432:5432 \
+       -e POSTGRES_USER=mayan \
+       -e POSTGRES_DB=mayan \
+       -e POSTGRES_PASSWORD=mayanuserpass \
+       -v /docker-volumes/mayan-edms/postgres:/var/lib/postgresql/data \
+       |DOCKER_POSTGRES_IMAGE_VERSION|
+
+   The PostgreSQL container will have one database named ``mayan``, with an user
+   named ``mayan`` too, with a password of ``mayanuserpass``. The container will
+   expose its internal 5432 port (PostgreSQL's default port) via the host's
+   5432 port. The data of this container will reside on the host's
+   ``/docker-volumes/mayan-edms/postgres`` folder.
+
+
+#. Create and run a Redis container
+
+   .. code-block:: bash
+
+       docker run \
+       -d \
+       --name mayan-edms-redis \
+       --restart=always \
+       -p 6379:6379
+       |DOCKER_REDIS_IMAGE_VERSION| \
+       redis-server \
+       --databases \
+       "2" \
+       --maxmemory-policy \
+       allkeys-lru \
+       --save \
+       ""
+
+   The Redis container will have two databases, one for background task messages,
+   and the other to hold the results of those background tasks. Redis is
+   configure to not save its content to disk and to automatically clear up
+   memory.
+
+#. Create and run a Mayan EDMS container
+
+   .. code-block:: bash
+
+       docker run \
+       -d \
+       --name mayan-edms \
+       --restart=always \
+       -p 80:8000 \
+       -e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'172.17.0.1'}}" \
+       -e MAYAN_CELERY_BROKER_URL="redis://172.17.0.1:6379/0" \
+       -e MAYAN_CELERY_RESULT_BACKEND="redis://172.17.0.1:6379/1" \
+       -v /docker-volumes/mayan-edms/media:/var/lib/mayan \
+       mayanedms/mayanedms:<version>
+
+   The Mayan EDMS container will connect to the PostgreSQL container via the
+   ``172.17.0.1`` IP address (the Docker host's default IP address). It will
+   connect using the ``django.db.backends.postgresql`` database driver and
+   connect to the ``mayan`` database using the ``mayan`` user with the password
+   ``mayanuserpass``. The files of the container will be store in the
+   host's ``/docker-volumes/mayan-edms/media`` folder. The container will
+   expose its web service running on port 8000 on the host's port 80.
+
+   The container will be available by browsing to ``http://localhost`` or to
+   the IP address of the computer running the container.
+
+   If another web server is running on port 80 use a different port in the
+   ``-p`` option. For example: ``-p 81:8000``.

docs/chapters/docker/nightly.txt

@@ -0,0 +1,12 @@
+==============
+Nightly images
+==============
+
+The continuous integration pipeline used for testing development builds also
+produces a resulting Docker image. These are build automatically and their
+stability is not guaranteed. They should never be used in production.
+If you want to try out the Docker images the development uses or want a sneak
+peek at the new features being worked on checkout the container registry at:
+https://gitlab.com/mayan-edms/mayan-edms/container_registry
+
+

docs/chapters/docker/start_stop.txt

@@ -0,0 +1,13 @@
+===================================
+Stopping and starting the container
+===================================
+
+To stop the container use::
+
+    docker stop mayan-edms
+
+
+To start the container again::
+
+    docker start mayan-edms
+

docs/chapters/docker/upgrading.txt

@@ -0,0 +1,30 @@
+=========
+Upgrading
+=========
+
+Upgrading a Mayan EDMS Docker container is actually a matter of stopping and
+deleting the container, downloading the most recent version of the image and
+starting a container again. The container will take care of updating the
+database structure to the newest version if necessary.
+
+**IMPORTANT!** Do not delete the volume storing the data, only the container.
+
+Stop the container to be upgraded::
+
+    docker stop mayan-edms
+
+
+Remove the container::
+
+    docker rm mayan-edms
+
+
+Pull the new image version::
+
+    docker pull mayanedms/mayanedms:latest
+
+
+Start the container again with the new image version::
+
+    docker run -d --name mayan-edms --restart=always -p 80:8000 -v /docker-volumes/mayan:/var/lib/mayan mayanedms/mayanedms:latest
+

docs/chapters/document_types.txt

@@ -1,3 +1,4 @@
+==============
 Document types
 ==============
 
@@ -67,22 +68,3 @@ assigned when uploaded into Mayan EDMS. A document can only be of one
 type at a given moment, but if needed, the type of a document can be changed.
 Upon changing its type, the document will lose its previous settings and
 attributes, and will inherit the settings and attributes of its new type.
-
-
-Creating document types
-=======================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Create document types" permission is required for this action.
-
-
-#. Go to the :menuselection:`System --> Setup --> Document types` menu.
-#. From the :guilabel:`Actions` dropdown select :guilabel:`Create document type`.
-#. Enter a label to be shown to users when using this document type.
-#. Optional: Enter a period of time after which documents of this type will be moved to the trash.
-#. Optional: Select the unit of time for the period after which the document will be moved to the trash.
-#. Optional: Enter a period of time after which documents moved to the trash will be permanently deleted.
-#. Optional: Select the unit of time for the period of time after which the documents moved to the trash will be permanently deleted
-#. Press :guilabel:`Submit`.

docs/chapters/documentation.txt

@@ -0,0 +1,24 @@
+=============
+Documentation
+=============
+
+The documentation is written in `reStructured Text`_ format, processed with
+Sphinx_, and resides in the ``docs`` directory. In order to build it, you will
+first need to install the documentation editing dependencies with::
+
+    $ pip install -r requirements/documentation.txt
+
+Then, to build an HTML version of the documentation, run the following command
+from the **docs** directory::
+
+    $ make docs-serve
+
+The generated documentation can be viewed by browsing to http://127.0.0.1:8000
+or by browsing to the ``docs/_build/html`` directory.
+
+You can also generate the documentation in formats other than HTML. Consult the
+Sphinx_ documentation for more details.
+
+.. _`reStructured Text`: http://docutils.sourceforge.net/rst.html
+.. _Sphinx: http://sphinx.pocoo.org
+

docs/chapters/file_storage.txt

@@ -1,6 +1,6 @@
-************
+============
 File storage
-************
+============
 
 The files are stored and placed under Mayan EDMS "control" to avoid
 filename clashes each file gets renamed to its ``UUID`` (Universally Unique ID),

docs/chapters/indexes/examples.txt

@@ -1,9 +1,9 @@
-
+==============
 Index examples
---------------
+==============
 
 Index of document types
-^^^^^^^^^^^^^^^^^^^^^^^
+=======================
 
 .. admonition:: Permissions required
     :class: warning
@@ -56,7 +56,7 @@ of types "Invoice" and "Letter", these will now go into the level "Correspondenc
 
 
 Index document by department, taken from the first character of the invoice number metadata
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+===========================================================================================
 
 Requires one index node with the template::
 
@@ -66,7 +66,7 @@ Requires one index node with the template::
 
 
 Nested date index from a date contained in a metadata
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+=====================================================
 
 Assuming the metadata type is named **date_issued** with a date format
 of YYYY-MM-DD. The target is to have two levels: one for years and another
@@ -91,7 +91,7 @@ sub level for months.
 
 
 Index by OCR content
-^^^^^^^^^^^^^^^^^^^^
+====================
 
 This example indexes documents in a "quarterly report" level if they have the
 fragment “quarterly report” in the OCR text::
@@ -105,21 +105,21 @@ The same applies to text content extracted for the document::
 
 
 Index documents not found in any cabinet
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+========================================
 ::
 
     {% if document.cabinets.count == 0 %}No Cabinets{% endif %}
 
 
 Index documents not tagged
-^^^^^^^^^^^^^^^^^^^^^^^^^^
+==========================
 ::
 
     {% if document.tags.count == 0 %}No Tags{% endif %}
 
 
 Index documents specifically, by the year of a metadata field otherwise by their uploaded year
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+==============================================================================================
 ::
 
     {% for tag in document.tags.all %}{% if tag.label == "Taxes" %}{% if document.metadata_value_of.tax_year|length_is:"4" %}{{ document.metadata_value_of.tax_year }}{% else %}{{ document.date_added|date:"Y" }}{% endif %}{% endif %}{% endfor %}

docs/chapters/indexes/index.txt

@@ -0,0 +1,23 @@
+.. index:: Indexes
+
+================
+Document indexes
+================
+
+Document indexes are a multi level structure similar to folder in a filesystem,
+however the hierarchy of the levels is automatically created, updated, or deleted
+based on rules set by the administrator.
+
+- :doc:`indexes`
+- :doc:`examples`
+- :doc:`mirroring`
+
+.. toctree::
+  :hidden:
+
+  indexes
+  mirroring
+  examples
+
+
+

docs/chapters/indexes/indexes.txt

@@ -1,10 +1,11 @@
-Indexes
+===============
-=======
+Index structure
+===============
 
 Indexes are an automatic method to hierarchically organize documents in
-relation to their properties (:doc:`../chapters/metadata`, label, MIME type,
+relation to their properties (:doc:`../metadata`, label, MIME type,
 etc). To use indexes you need to first create an index template. Once created,
-associate the index to one or more :doc:`../chapters/document_types`.
+associate the index to one or more :doc:`../document_types`.
 
 Index are hierarchical models so a tree template needs to be specified for them.
 This tree template will contain references to document metadata or properties
@@ -75,41 +76,3 @@ that will be generate based on the tree template would be as follows:
       year_1 -> document_3;
 
    }
-
-.. include:: ../chapters/index_examples.rst
-
-
-Mirroring
----------
-
-Indexes can be exported as `FUSE <https://en.wikipedia.org/wiki/Filesystem_in_Userspace>`_
-filesystems. Using the management command ``mountindex`` we could export the
-previous example index as follows::
-
-    mkdir -p ~/indexes/products
-    mayan-edms.py mountindex product-sheets-per-year ~/indexes/products
-
-The ``~/indexes/products`` directory will now have a directory and files structure
-identical to that of the index. Once indexes are mounted with this command, they
-behave like any other filesystem directory and can even be further shared
-via the network with network file system software like
-`Samba <https://www.samba.org/>`_ or
-`NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.
-
-.. blockdiag::
-
-    blockdiag {
-        orientation = portrait
-        span_width = 200;
-
-        index [ label = 'Product sheets per year', width=180 ];
-        block_device [ height = 100, label = "Block device\n(Hard drive)", shape = flowchart.database ];
-        network [ label = "Network", shape = cloud ];
-        user [ label = "Users", shape = actor ];
-
-        index -> block_device [ label = "mirroring", fontsize = 8 ];
-        block_device -> network -> user;
-   }
-
-Indexes and mirrored indexes are Read Only as they are generated as a result of
-prior activities like document uploads, metadata changes.

docs/chapters/indexes/mirroring.txt

@@ -0,0 +1,35 @@
+=========
+Mirroring
+=========
+
+Indexes can be exported as `FUSE <https://en.wikipedia.org/wiki/Filesystem_in_Userspace>`_
+filesystems. Using the management command ``mountindex`` we could export the
+previous example index as follows::
+
+    mkdir -p ~/indexes/products
+    mayan-edms.py mountindex product-sheets-per-year ~/indexes/products
+
+The ``~/indexes/products`` directory will now have a directory and files structure
+identical to that of the index. Once indexes are mounted with this command, they
+behave like any other filesystem directory and can even be further shared
+via the network with network file system software like
+`Samba <https://www.samba.org/>`_ or
+`NFS <https://en.wikipedia.org/wiki/Network_File_System>`_.
+
+.. blockdiag::
+
+    blockdiag {
+        orientation = portrait
+        span_width = 200;
+
+        index [ label = 'Product sheets per year', width=180 ];
+        block_device [ height = 100, label = "Block device\n(Hard drive)", shape = flowchart.database ];
+        network [ label = "Network", shape = cloud ];
+        user [ label = "Users", shape = actor ];
+
+        index -> block_device [ label = "mirroring", fontsize = 8 ];
+        block_device -> network -> user;
+   }
+
+Indexes and mirrored indexes are Read Only as they are generated as a result of
+prior activities like document uploads, metadata changes.

docs/chapters/languages.txt

@@ -1,6 +1,6 @@
-*********
+=========
 Languages
-*********
+=========
 
 The list of languages choices in the language dropdown used for documents is
 based on the current ISO 639 list. This list can be quite extensive. To reduce
@@ -31,6 +31,6 @@ Example::
 
 For more information check out the
 :ref:`environment variables <environment_variables>` chapter of the
-:doc:`../topics/settings` topic.
+:doc:`../parts/settings` topic.
 
 

docs/chapters/mailing.txt

@@ -1,12 +1,12 @@
-*****************
+=================
 Mailing documents
-*****************
+=================
 
 Sending emails in Mayan EDMS is controlled by two different system depending on
 the type of email being sent. These are administrative emails like password
 reset emails and user emails sent from the application. To configure
 administrative email for things like password reset check the topic:
-:doc:`../topics/administration`
+:doc:`../parts/administration`
 
 Application emails
 ==================
@@ -14,8 +14,7 @@ Application emails
 To allow users to send emails or documents from within the web interface,
 Mayan EDMS provides its our own email system called Mailing Profiles.
 Mailing Profiles support access control per user role and can use different
-email backends. Mailing Profiles are created from the
+email backends.
-:menuselection:`System --> Setup` menu.
 
 Once created mailing profiles allow users to send email messages from
 within the user interface containing either an URL link to the document or

docs/chapters/metadata.rst

@@ -1,99 +0,0 @@
-********
-Metadata
-********
-
-Metadata is the name of the attribute of a document. The concept of metadata is
-divided in two: **metadata types** (size, color, distance) and **metadata values** for
-those types. Metadata types are defined in the setup menu and associated with
-document types. Then when a document is uploaded, a value for that metadata
-can be entered. There are two kinds of metadata type to document type relations:
-optional and required. When a metadata type is optional for a document type,
-it can be left blank for a document being uploaded and the upload will still
-be successful. On the other hand required metadata type must be given a value
-or it will not be possible to upload the document at hand.
-
-Examples of metadata type: Invoice number, color, employee id.
-
-The data entry of metadata types can be set to allow any value to be provided
-(the default) or a list of possible values can be entered in the ``Lookup``
-configuration option and users will be presented with a drop down list of options
-instead of the default text entry box.
-
-If metadata types are setup to allow any value to be entered a ``validation``
-option can be chosen to block the entry of invalid data. Metadata types also
-provide ``parsers`` which will not block the entry of data but are able to
-interpret and modify the value provided by the user to a conform to a specific
-format. An example of a provided parser is the date parser which will interpret
-and correct dates provided by users regardless of the format in which they are
-entered.
-
-
-Creating metadata types
-=======================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Create new metadata types" permission is required for this action.
-
-
-#. Go to the :menuselection:`System --> Setup --> Metadata types` menu.
-#. From the :guilabel:`Actions` dropdown select :guilabel:`Create new`.
-#. Provide a name to reference this metadata type in other parts of the system.
-#. Enter a label to be shown to users when using this metadata type.
-#. Optional: Enter a default value for the metadata type.
-#. Optional: Provide a comma separated list of options to restrict the data entry
-   when using this metadata type.
-#. Optional: Select a validator and a parser to validate and cleanup the data
-   entry when not using a predetermined list of values.
-#. Press :guilabel:`Submit`.
-
-
-Assigning a metadata type to a document type
-============================================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    - The "Edit metadata types" permission is required for this action, globally or
-      via an ACL for a metadata type.
-    - Also the "Edit document type" permission
-      is required, globally or via an ACL for a document type.
-
-
-This action can be performed in two ways.
-
-Option 1: Via the metadata type view
-------------------------------------
-
-#. Go to the :menuselection:`System --> Setup --> Metadata types` menu.
-#. Click on the button :guilabel:`Document types` of the metadata type you which
-   to associate.
-#. From the list of existing document types press either:
-
-   - :guilabel:`None` if this metadata type will not be available for documents
-     of the type.
-   - :guilabel:`Optional` if this metadata type will be available and is
-     optional to provide a value for documents of the type.
-   - :guilabel:`Required` if this metadata type will be available and is
-     required to provide a value for documents of the type.
-
-#. Press :guilabel:`Save`.
-
-
-Option 2: Via the document type view
-------------------------------------
-
-#. Go to the :menuselection:`System --> Setup --> Document types` menu.
-#. Click on the button :guilabel:`Metadata types` of the metadata type you which
-   to associate.
-#. From the list of existing metadata types press either:
-
-   - :guilabel:`None` if this metadata type will not be available for documents
-     of the type.
-   - :guilabel:`Optional` if this metadata type will be available and is
-     optional to provide a value for documents of the type.
-   - :guilabel:`Required` if this metadata type will be available and is
-     required to provide a value for documents of the type.
-
-#. Press :guilabel:`Save`.

docs/chapters/metadata.txt

@@ -0,0 +1,28 @@
+========
+Metadata
+========
+
+Metadata is the name of the attribute of a document. The concept of metadata is
+divided in two: **metadata types** (size, color, distance) and **metadata values** for
+those types. Metadata types are defined in the setup menu and associated with
+document types. Then when a document is uploaded, a value for that metadata
+can be entered. There are two kinds of metadata type to document type relations:
+optional and required. When a metadata type is optional for a document type,
+it can be left blank for a document being uploaded and the upload will still
+be successful. On the other hand required metadata type must be given a value
+or it will not be possible to upload the document at hand.
+
+Examples of metadata type: Invoice number, color, employee id.
+
+The data entry of metadata types can be set to allow any value to be provided
+(the default) or a list of possible values can be entered in the ``Lookup``
+configuration option and users will be presented with a drop down list of options
+instead of the default text entry box.
+
+If metadata types are setup to allow any value to be entered a ``validation``
+option can be chosen to block the entry of invalid data. Metadata types also
+provide ``parsers`` which will not block the entry of data but are able to
+interpret and modify the value provided by the user to a conform to a specific
+format. An example of a provided parser is the date parser which will interpret
+and correct dates provided by users regardless of the format in which they are
+entered.

docs/chapters/object_storage.rst

@@ -1,31 +0,0 @@
-**************
-Object storage
-**************
-
-It is possible to use object storage instead of the default filesystem based
-storage. One such object storage system is Amazon S3 (Simple Storage Service)
-API compatible object storage. The following steps will configure Mayan EDMS
-to use a S3 style storage for documents.
-
-1. Install the django-storages and boto3 Python libraries.
-
-  * For the direct deployment method of installation use::
-
-        pip install django-storages boto3
-
-
-  * or if using the Docker image, add the following the command line that runs the container::
-
-    -e MAYAN_PIP_INSTALLS='django-storages boto3'
-
-
-2. From the web interface navigate to the :menuselection:`System --> Setup --> Setting --> Documents` menu.
-3. Locate the **DOCUMENTS_STORAGE_BACKEND** setting, press **Edit** and enter::
-
-    storages.backends.s3boto3.S3Boto3Storage
-
-4. Save and locate the setting **DOCUMENTS_STORAGE_BACKEND_ARGUMENTS**, press **Edit** and enter::
-
-    '{access_key: <your S3 access key>, secret_key: <your S3 secret key>, bucket_name: <S3 bucket name>}'
-
-5. Save and restart your Mayan EDMS installation for the setting to take effect.

docs/chapters/ocr_backend.txt

@@ -1,6 +1,6 @@
-***********
+===========
 OCR backend
-***********
+===========
 
 Mayan EDMS ships an OCR backend that uses the FLOSS engine Tesseract
 (https://github.com/tesseract-ocr/tesseract/), but it can

docs/chapters/password_reset.txt

@@ -1,6 +1,6 @@
-**************
+==============
 Password reset
-**************
+==============
 
 To use the password reset feature, administrative emails need to be configured.
 These are sent by the system itself and not by the users. Their usage and
@@ -27,7 +27,7 @@ Example::
 To change the reference URL in the password reset emails on in the
 default document mailing template modify the ``COMMON_PROJECT_URL`` setting.
 For information on the different ways to change a setting check the
-:doc:`../topics/settings` topic.
+:doc:`../parts/settings` topic.
 
 To test the email settings use the management command ``sendtestemail``.
 Example::

docs/chapters/permissions.txt

@@ -1,6 +1,6 @@
-***********
+===========
 Permissions
-***********
+===========
 
 Mayan EDMS provides very fine control over which actions users can
 perform. Action control works by allowing ``roles``, that are composed of

docs/chapters/philosophies.txt

@@ -0,0 +1,29 @@
+====================
+Project philosophies
+====================
+
+How to think about Mayan EDMS when doing changes or adding new features;
+why things are the way they are in Mayan EDMS:
+
+- Functionality must be as market/sector independent as possible, code for the
+  95% of use cases.
+- Each user must be able to configure and customize it to their needs after
+  install.
+- Abstract as much as possible, each app must be an expert in just one thing,
+  for other things they should use the API/classes/functions of other apps.
+- Assume as little as possible about anything outside the project
+  (hardware, OS, storage).
+- Provide Python based abstraction so that a default install runs with a single
+  step.
+- No hard dependencies on binaries unless there is no other choice.
+- Provide “drivers” or switchable backends to allow users to fine tune the
+  installation.
+- Call to binaries only when there is no other choice or the Python choices are
+  not viable/mature/efficient.
+- Each app is as independent and self contained as possible. Exceptions, the
+  basic requirements: navigation, permissions, common, main.
+- If an app is meant to be used by more than one other app, it should be as
+  generic as possible in regard to the project and another app will bridge the functionality.
+
+  - Example: since indexing (document_indexing) only applies to documents, the
+    app is specialized and depends on the documents app.

docs/chapters/quick_labels.rst

@@ -1,70 +0,0 @@
-************
-Quick labels
-************
-
-Quick labels are predetermined filenames that allow the quick renaming of
-documents as they are uploaded or after they have been uploaded.
-
-Quick labels are added and associated to a document type.
-
-Example of quick label: Invoice, Receipt from X store, Purchase order.
-
-It is possible to preserve the file extension when using quick labels.
-Extensions are required for some operating system to be able to detect the
-correct file type to access the content.
-
-For example if a document file is named "file0001.pdf" and the quick label
-"Receipt from X store" is applied, the resulting document label will be
-"Receipt from X store.pdf".
-
-
-Creating quick labels
-=====================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Edit document types" permission is required for this action, either
-    globally or via an ACL for a document type.
-
-
-Since quick labels are associated with document types, creating quick labels
-must be done from the document type view.
-
-#. Go to the :menuselection:`System --> Setup --> Document types` menu.
-#. In the document type list, click on the :guilabel:`Quick labels` button of
-   the document type for which you wish to create a quick label.
-#. In the view titled "Quick labels for document type: <your document type>",
-   from the :guilabel:`Actions` dropdown select :guilabel:`Add quick label to document type`.
-#. At the quick label creation form enter the desired label and press :guilabel:`Save`.
-
-
-Using quick labels during upload
-================================
-
-#. Use the new document upload wizard from :menuselection:`Documents --> New document`.
-#. Select a document type and navigate to the penultimate step, where you have
-   the option to drag and drop files to upload.
-#. Select a an option from the :guilabel:`Quick document rename` dropdown.
-#. Optionally select the :guilabel:`Preserve extension` checkbox to keep the file
-   extension.
-#. Upload your documents.
-
-
-Using quick labels for existing documents
-=========================================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Edit document properties" permission is required for this action, either
-    globally or via an ACL for a document or document type.
-
-
-#. Navigate to the document preview view of the document to rename. Make sure
-   quick labels for the document type of the document select have been created.
-#. From the :guilabel:`Actions` dropdown select :guilabel:`Edit Properties`.
-#. Select a an option from the :guilabel:`Quick document rename` dropdown.
-#. Optionally select the :guilabel:`Preserve extension` checkbox to keep the file
-   extension.
-#. Press :guilabel:`Save` to rename the document.

docs/chapters/quick_labels.txt

@@ -0,0 +1,18 @@
+============
+Quick labels
+============
+
+Quick labels are predetermined filenames that allow the quick renaming of
+documents as they are uploaded or after they have been uploaded.
+
+Quick labels are added and associated to a document type.
+
+Example of quick label: Invoice, Receipt from X store, Purchase order.
+
+It is possible to preserve the file extension when using quick labels.
+Extensions are required for some operating system to be able to detect the
+correct file type to access the content.
+
+For example if a document file is named "file0001.pdf" and the quick label
+"Receipt from X store" is applied, the resulting document label will be
+"Receipt from X store.pdf".

docs/chapters/releases.txt

@@ -0,0 +1,162 @@
+===================
+Installable package
+===================
+
+Source file package
+-------------------
+
+This is the sequence of step used to produce an installable package:
+
+1. Generate the packaged version (will produce dist/mayan-edms-x.y.z.tar.gz)::
+
+    $ make sdist
+
+2. Do a test install::
+
+    $ cd /tmp
+    $ virtualenv venv
+    $ source venv/bin/activate
+    $ pip install <path of the Git repository>/dist/mayan-edms-x.y.z.tar.gz
+    $ mayan-edms.py initialsetup
+    $ mayan-edms.py runserver
+
+
+Wheel package
+-------------
+
+1. Install the development requirements::
+
+    $ pip install -r requirements/development.txt
+
+2. Create wheel package using the makefile::
+
+    $ make wheel
+
+3. Do a test install::
+
+    $ cd /tmp
+    $ virtualenv venv
+    $ source venv/bin/activate
+    $ pip install <path of the Git repository>/dist/mayan_edms-x.y.z-py2-none-any.whl
+    $ mayan-edms.py initialsetup
+    $ mayan-edms.py runserver
+
+
+Version numbering
+=================
+
+Mayan EDMS uses the Semantic Versioning (http://semver.org/) method to choose
+version numbers along with Python's PEP-0440 (https://www.python.org/dev/peps/pep-0440/)
+to format them.
+
+X.YaN   # Alpha release
+X.YbN   # Beta release
+X.YrcN  # Release Candidate
+X.Y     # Final release
+
+
+Release checklist
+=================
+
+#. Check for missing migrations::
+
+    make check-missing-migrations
+
+#. Synchronize translations::
+
+    make translations-pull
+
+#. Compile translations::
+
+    make translations-compile
+
+#. Update changelog.
+#. Write release notes.
+#. Scan the code with flake8 for simple style warnings.
+#. Check README.rst format with::
+
+    python setup.py check -r -s
+
+   or with::
+
+       make check-readme
+
+#. Bump version in ``mayan/__init__.py`` and ``docker/rootfs/version``::
+
+    make increase-version PART=<major, minor or micro>
+
+#. Update requirements version in ``setup.py`` using:
+   ::
+
+       make generate-setup
+
+#. Commit as version bump.
+#. Build source package and test:
+   ::
+
+       make test-sdist-via-docker-ubuntu
+
+#. Build wheel package and test:
+   ::
+
+       make test-wheel-via-docker-ubuntu
+
+#. Tag version:
+   ::
+
+       git tag -a vX.Y.Z -m "Version X.Y.Z"
+
+#. Generate set ``setup.py`` again to update the build number::
+
+    make generate-setup
+
+#. Commit the new ``setup.py`` file.
+
+#. Release the version using one of the two following methods: GitLab CI or
+   manual
+
+Release using GitLab CI
+-----------------------
+
+#. Push the current brach:
+   ::
+
+       git push
+
+
+#. Push the new tags:
+   ::
+
+       git push --tags
+
+
+#. Delete the corresponding ``releases/`` branch:
+   ::
+
+       git push origin :releases/<branch>
+
+#. Push the current branch to the corresponding origin ``releases/`` branch:
+   ::
+
+       git push origin <local branch>:releases/<branch>
+
+
+Manual release
+--------------
+
+#. Build and upload a test release:
+   ::
+
+       make release-test-via-docker-ubuntu
+
+#. Build and upload a final release:
+   ::
+
+       make release-via-docker-ubuntu
+
+Other steps
+-----------
+
+#. Update the contrib/scripts/install/docker.sh values
+
+#. Upload contrib/scripts/install/docker.sh to https://get.mayan-edms.com

docs/chapters/rest_api.txt

@@ -1,6 +1,6 @@
-***
+========
-API
+REST API
-***
+========
 
 Mayan EDMS provides an HTTP REST Application Program Interface (or API). This
 API allows integration with 3rd party software using simple HTTP requests.

docs/chapters/scaling_up.rst

@@ -1,155 +0,0 @@
-**********
-Scaling up
-**********
-
-The default installation method fits most use cases. If you use case requires
-more speed or capacity here are some suggestion that can help you improve the
-performance of your installation.
-
-
-Change the database manager
-===========================
-
-Use PostgreSQL or MySQL as the database manager.
-Tweak the memory setting of the database manager to increase memory allocation.
-More PostgreSQL specific examples are available in their wiki page:
-https://wiki.postgresql.org/wiki/Performance_Optimization
-
-
-Increase the number of Gunicorn workers
-=======================================
-
-The Gunicorn workers process HTTP requests and affect the speed at which the
-website responds.
-
-If you are using the Docker image, change the value of the
-``MAYAN_GUNICORN_WORKERS`` environment variable (check the Docker image chapter:
-:ref:`docker_environment_variables`). Normally this variable defaults to 2.
-Increase this number to match the number of CPU cores + 1.
-
-If you are using the direct deployment methods, change the line that reads::
-
-    command = |MAYAN_GUNICORN_BIN| -w 2 mayan.wsgi --max-requests 500 --max-requests-jitter 50 --worker-class gevent --bind 0.0.0.0:8000 --timeout 120
-
-And increase the value of the ``-w 2`` argument. This line is found in the
-``[program:mayan-gunicorn]`` section of the supervisor configuration file.
-
-
-Background task processing
-==========================
-
-The Celery workers are system processes that take care of the background
-tasks requested by the frontend interactions like document image rendering
-and periodic tasks like OCR. There are several dozen tasks defined in the code.
-These tasks are divided into queues based on the app of the relationship
-between the tasks. The queues by default are divided into three groups
-based on the speed at which they need to be processed. The document page
-image rendering for example is categorized as a high volume, short duration
-task. The OCR is a high volume, long duration task. Email checking is a
-low volume, medium duration tasks. It is not advisable to have the same
-worker processing OCR to process image rendering too. If the worker is
-processing several OCR tasks it will not be able to provide fast images
-when an user is browsing the user interface. This is why by default the
-queues are split into 3 workers: fast, medium, and slow. Each worker will handle
-queues based on the latency required by each queue group.
-
-
-Optimizations
--------------
-
-* Increase the number of workers and redistribute the queues among them
-  (only possible with direct deployments).
-* Launch more workers to service a queue. For example for faster document
-  image generation launch 2 workers to process the converter queue only
-  possible with direct deployments).
-* By default each worker process uses 1 thread. You can increase the thread
-  count of each worker process with the Docker environment options:
-
-  * ``MAYAN_WORKER_FAST_CONCURRENCY``
-  * ``MAYAN_WORKER_MEDIUM_CONCURRENCY``
-  * ``MAYAN_WORKER_SLOW_CONCURRENCY``
-
-* If using direct deployment, increase the value of the ``--concurrency=1``
-  argument of each worker in the supervisor file. You can also remove this
-  argument and let the Celery algorithm choose the number of threads to
-  launch. Usually this defaults to the number of CPU cores + 1.
-
-
-Change the message broker
-=========================
-Messages are the method of communication between front end interactive code
-and background tasks. In this regard messages can be thought as homologous
-to tasks requests. Improving how many messages can be sent, stored and
-sorted will impact the number of tasks the system can handle. To save on
-memory, the basic deployment method and the Docker image default to using
-Redis as a message broker. To increase capacity and reduce volatility of
-messages (pending tasks are not lost during shutdown) use RabbitMQ to
-shuffle messages.
-
-For direct installs refer to the :ref:`deployment_advanced` documentation
-section for the required changes.
-
-For the Docker image, launch a separate RabbitMQ container
-(https://hub.docker.com/_/rabbitmq/)::
-
-    docker run -d --name mayan-edms-rabbitmq -e RABBITMQ_DEFAULT_USER=mayan -e RABBITMQ_DEFAULT_PASS=mayanrabbitmqpassword -e RABBITMQ_DEFAULT_VHOST=mayan |DOCKER_RABBITMQ_IMAGE_VERSION|
-
-Pass the MAYAN_CELERY_BROKER_URL environment variable (https://kombu.readthedocs.io/en/latest/userguide/connections.html#connection-urls)
-to the Mayan EDMS container so that it uses the RabbitMQ container the
-message broker::
-
-    -e MAYAN_CELERY_BROKER_URL="amqp://mayan:mayanrabbitmqpassword@localhost:5672/mayan",
-
-When tasks finish, they leave behind a return status or the result of a
-calculation, these are stored for a while so that whoever requested the
-background task, is able retrieve the result. These results are stored in the
-result storage. By default a Redis server is launched inside the Mayan EDMS
-container. You can launch a separate Docker Redis container and tell the Mayan
-EDMS container to use this via the ``MAYAN_CELERY_RESULT_BACKEND`` environment
-variable. The format of this variable is explained here: http://docs.celeryproject.org/en/3.1/configuration.html#celery-result-backend
-
-
-Deployment type
-===============
-
-Docker provides a faster deployment and the overhead is not high on modern
-systems. It is however memory and CPU limited by default and you need to
-increase this limits. The settings to change the container resource limits
-are here: https://docs.docker.com/config/containers/resource_constraints/#limit-a-containers-access-to-memory
-
-For the best performance possible use the advanced deployment method on a
-host dedicated to serving only Mayan EDMS.
-
-
-Storage
-=======
-
-For best input and output speed use a block based local filesystem on an
-SSD drive for the ``/media`` sub folder. The location of the ``/media`` folder
-will be specified by the ``MEDIA_ROOT`` setting.
-
-If capacity is your bottom line, switch to an
-:doc:`object storage <../chapters/object_storage>` system.
-
-
-Use additional hosts
-====================
-
-When one host is not enough you can use multiple hosts and share the load.
-Make sure that all hosts share the ``/media`` folder as specified by the
-``MEDIA_ROOT`` setting, also the database, the broker, and the result storage.
-One setting that needs to be changed in this configuration is the lock
-manager backend.
-
-Resource locking is a technique to avoid two processes or tasks to modify
-the same resource at the same time causing a race condition. Mayan EDMS uses
-its own lock manager. By default the lock manager with use a simple file
-based lock backend ideal for single host installations. For multiple hosts
-installation the database backend must be used in other to coordinate the
-resource locks between the different hosts over a share data medium. This is
-accomplished by modifying the environment variable ``LOCK_MANAGER_BACKEND`` in
-both the direct deployment or the Docker image. Use the value
-``lock_manager.backends.model_lock.ModelLock`` to switch to the database
-resource lock backend. If you can also write your own lock manager backend
-for other data sharing mediums with better performance than a relational
-database like Redis, Memcached, Zoo Keeper.

docs/chapters/settings.txt

@@ -1,6 +1,6 @@
-********
+========
 Settings
-********
+========
 
 Mayan EDMS can be configure via environment variables or by setting files.
 
@@ -14,7 +14,7 @@ To use environment variables, lookup the name of the setting you want to
 override in the "Settings" menu. The "Settings" menu is located inside the
 "Setup" main menu. To pass a value via an environment variable append
 ``"MAYAN_"`` to the name of the settings option. For example, to change
-the number of documents displayed per page (COMMON_PAGINATE_BY, by default 40),
+the number of documents displayed per page (``COMMON_PAGINATE_BY``, by default 40),
 use::
 
     export MAYAN_COMMON_PAGINATE_BY=10
@@ -32,8 +32,8 @@ Via YAML configuration file
 .. versionadded:: 3.1
 
 It is possible to modify the different settings by creating or editing the
-``media/config.yml`` file. This file is formatted in the YAML markup language (
+``media/config.yml`` file. This file is formatted in the YAML markup language
-http://yaml.org/). Here is an example of what the looks like::
+(http://yaml.org/). Here is an example of what the looks like::
 
     DOCUMENT_PARSING_AUTO_PARSING: true
     DOCUMENT_PARSING_PDFTOTEXT_PATH: /usr/bin/pdftotext
@@ -69,7 +69,7 @@ documentation your ``mayan_settings`` folder should be located in the directory:
 
 If Mayan EDMS was installed using Docker, the ``mayan_settings`` folder
 will be found inside the install Docker volume. If you installed Mayan EDMS
-according to the :doc:`../chapters/docker` instructions provided in this
+according to the :doc:`../chapters/docker/index` instructions provided in this
 documentation your ``mayan_settings`` folder should be located in the directory:
 ``/docker-volumes/mayan/mayan_settings``.
 
@@ -82,7 +82,7 @@ Create a file with any valid name and a ``.py`` extension in the
 Now add the corresponding lines to override the default settings.
 In the settings file, it is not necessary to prepend the string ``MAYAN_`` to
 the setting option. For example, to change the number of documents displayed
-per page (COMMON_PAGINATE_BY, by default 40),
+per page (``COMMON_PAGINATE_BY``, by default 40),
 use::
 
     COMMON_PAGINATE_BY=10
@@ -110,16 +110,16 @@ or via the ``DJANGO_SETTINGS_MODULE`` environment variable like this::
 
 To make the use of the custom Python setting file permanent, update the ``|MAYAN_SUPERVISOR_CONF|``
 file. In the ``[supervisord]`` section, update the ``environment=`` value of
-``DJANGO_SETTINGS_MODULE`` from  the default ``mayan.settings.production`` to 
+``DJANGO_SETTINGS_MODULE`` from  the default ``mayan.settings.production`` to
 ``mayan.media.mayan_settings.mysettings``. Pay attention to the indentation of
 the ``environment=`` entries.
 
 Force supervisor to read the changes and restart using::
-    
+
     sudo supervisorctl reread
     sudo supervisorctl restart all
 
-For the :doc:`../chapters/docker` installation method, the full import path will be
+For the :doc:`../chapters/docker/index` installation method, the full import path will be
 ``mayan_settings.mysettings`` and can only be passed via the
 ``MAYAN_SETTINGS_MODULE`` environment variable like this::
 

docs/chapters/signatures.txt

@@ -1,6 +1,6 @@
-*******************
+===================
 Document signatures
-*******************
+===================
 
 Mayan EDMS supports two types of document signatures: embedded and
 detached signatures. When a document with an embedded signature is

docs/chapters/smart_links.txt

@@ -1,3 +1,4 @@
+===========
 Smart links
 ===========
 
@@ -6,7 +7,7 @@ in their respective indexes. Smart links are useful when two documents are
 related somehow but are of different type or different hierarchical units.
 
 Example: A patient record can be related to a prescription drug information
-document, but they each belong to their own :doc:`../chapters/indexes`.
+document, but they each belong to their own :doc:`../chapters/indexes/index`.
 
 Smart links are rule based, but don't create any organizational structure.
 Smart links just show the documents that match the rules as evaluated against

docs/chapters/source_control.txt

@@ -0,0 +1,102 @@
+==============
+Source Control
+==============
+
+Mayan EDMS source is controlled with Git_.
+
+The project is publicly accessible, hosted and can be cloned from **GitLab** using::
+
+    $ git clone https://gitlab.com/mayan-edms/mayan-edms.git
+
+
+Git branch structure
+--------------------
+
+Mayan EDMS follows a simplified model layout based on Vincent Driessen's
+`Successful Git Branching Model`_ blog post.
+
+``/versions/micro``
+    Working branch for the next bugfix release. Micro increment (third digit).
+    Only bug fixes, minor features, back-ported urgent features. This
+    branch is stable and safe for production.
+``/versions/minor``
+    Working branch for the next minor release (second digit). New features,
+    occasional breakage. Not for production but should run in test
+    environment most of the time. This is the branch you will want to
+    try out if you want to check out new features.
+``/versions/major``
+    Working branch for the next major release (first digit). New features,
+    incompatible changes to the user facing interfaces. Broken most of the
+    time, not for production and should only be cloned by developers
+    with experience with Mayan's development.
+``master``
+    Current production release (|version|). Points to the latest version of
+    the latest series. Production quality code.
+``features/``
+    Working branches for unfinished and unmerged feature. Likely unstable,
+    don't use in production. Once the feature is complete, it is merged
+    into one of the versions branches and deleted.
+
+Special branches:
+
+``releases/all``
+    Pushing code to this branch will trigger the build and release
+    a new Docker image, Documentation and Python package.
+``releases/docker``
+    Pushing code to this branch will trigger the build and release
+    of a new Docker image to Docker Hub.
+``releases/documentation``
+    Pushing code to this branch will trigger the build and release
+    of new documentation.
+``releases/python``
+    Pushing code to this branch will trigger the build and release
+    of a new Python package to PyPI.
+``nightly``
+    Pushing code to this branch will trigger the build and release
+    of a new Docker image based on development code to the GitLab image
+    repository only. The image will not be published to Docker Hub.
+
+Each release is tagged separately using annotated Git tags.
+
+When submitting patches, please place your code in its own ``feature/`` branch
+prior to opening a Merge Request on GitLab_.
+
+.. _Git: http://git-scm.org
+.. _`Successful Git Branching Model`: http://nvie.com/posts/a-successful-git-branching-model/
+
+
+Commit messages
+---------------
+
+#. Use English as the language for the commit messages.
+#. Provide a subject line composed of a tag and a short explanation::
+
+    Indexing: Add document base property reindex
+
+#. Keep the subject line to 50 or less characters.
+#. Capitalize the subject line.
+#. Don't end the subject line with a period, leave like a phrase in English.
+#. Use active voice in the. Say what the commit will do when applied not what
+   you did::
+
+       Add new properties to the model.
+
+   Vs.
+   ::
+
+       Added new properties to the model.
+
+#. Limit the body of the commit to 72 characters.
+#. When a commit fixes or improves an issue add the issue number in the commit
+   message. Either in the subject or in the body.
+#. Sign commit messages.
+#. Use explicit language even for minor commits. Don't do::
+
+       Fix typo
+
+   Use::
+
+       Document: Fix typo in label description
+
+
+.. _GitLab: https://gitlab.com/mayan-edms/mayan-edms/

docs/chapters/sources.txt

@@ -1,3 +1,4 @@
+=======
 Sources
 =======
 
@@ -62,19 +63,3 @@ manager system.
 
     If you deployed a Mayan EDMS Docker container and want to use watched folders
     or staging folder, refer to the Docker chapter :ref:`docker-accessing-outside-data`.
-
-
-Creating new sources
-====================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Create new document sources" permission is required for this action.
-
-
-#. Go to the :menuselection:`System --> Setup --> Sources` menu.
-#. From the :guilabel:`Actions` dropdown select the new type of source to create.
-#. Each source type will have different fields to customize its behavior. Enter
-   the required information in each field based on the help text provided.
-#. Press :guilabel:`Save`.

docs/chapters/tags.rst

@@ -1,46 +0,0 @@
-Tags
-====
-
-Tags are color coded properties that can be attached or removed from documents.
-Tags allow giving documents a binary property. Documents can also be tagged
-with more than one tag.
-
-Once tagged, documents can be searched by their tags. It is also possible to
-show all the documents tagged with a particular
-tag via the **Documents** link of each tag.
-
-To view, add, or remove tags from a specific document, go to the document view
-and click on the **Tags** tab. This view will show the current tags of a
-document. The **Actions** menu will now show two new options: one to attach
-more tags to the document and another to remove tags from the document.
-
-It is also possible to tag documents in bulk by selecting several documents
-from any view that displays a list documents and selecting the **Attach tag**
-or **Remove tags** from the dropdown menu.
-
-
-Creating tags
-=============
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Create new tags" permission is required for this action.
-
-
-#. Go to the :menuselection:`Tags --> Create` menu.
-#. Enter a label to identify the tag.
-#. Select a color for the tag.
-#. Press :guilabel:`Submit`.
-
-
-View all tags
-=============
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "View tags" permission is required for this action, globally or via an
-    ACL for a tag.
-
-#. Go to the :menuselection:`Tags --> All` menu.

docs/chapters/tags.txt

@@ -0,0 +1,11 @@
+====
+Tags
+====
+
+Tags are color coded properties that can be attached or removed from documents.
+Tags allow giving documents a binary property. Documents can also be tagged
+with more than one tag.
+
+Once tagged, documents can be searched by their tags. It is also possible to
+show all the documents tagged with a particular
+tag via the **Documents** link of each tag.

docs/chapters/transformations.txt

@@ -1,6 +1,6 @@
-***************
+===============
 Transformations
-***************
+===============
 
 Transformations are persistent manipulations to the previews of the stored
 documents. For example: a scanning equipment may only produce landscape PDFs.
@@ -17,5 +17,5 @@ menu will have a new option that reads "Create new transformation". Currently,
 the available transformations are: rotation, zoom, crop, and resize. Once the
 document image has been corrected, resubmit it for OCR for improved results.
 
-Transformations are not destructive and do not physically modify the document
+Transformations are non destructive and do not physically modify the document
 file, they just modify the document's graphical representation.

docs/chapters/translations.txt

@@ -1,6 +1,6 @@
-************
+============
 Translations
-************
+============
 
 Translations are handled online via the **Transifex** website:
 https://www.transifex.com/projects/p/mayan-edms/. To create a translation team

docs/chapters/upload_wizard.rst

@@ -1,84 +0,0 @@
-*************
-Upload wizard
-*************
-
-The steps needed to upgrade a document using form-tools' ``SessionWizard``
-were hardcoded in the ``source`` app. This made it very difficult to add or remove
-wizard steps.
-
-The steps of the wizard are now defined by a new class called
-``sources.wizard.WizardStep``. The existing steps to select a document type,
-enter metadata and tag the document, have been converted to function as
-``WizardSteps`` subclasses. The converted steps now live in
-
-``sources.wizards.WizardStepDocumentType``, ``tag.wizard_steps.WizardStepTags``,
-and ``metadata.wizard_steps.WizardStepMetadata``.
-
-The steps need to define the following methods:
-
-- ``done``: This method is execute when the wizard finished the last step
-  an enter the step where the actual file are uploaded. This steps is used
-  to encode form data into the URL query string that will be passed to the
-  document upload view for each file uploaded.
-
-- ``condition``: This method is used to display the step conditionally.
-  If this method return True it will be displayed during the upload wizard
-  execution. To skip the step, return False or None.
-
-- ``get_form_initial``: This method is used to return the initial data
-  for the step form. Use this method to set up initial values for the step's
-  form fields.
-
-- ``step_post_upload_process``: This method will be executed once the
-  document finishes uploading. Use this method to process the information
-  encoded in the URL querystring by the step's `done`` method.
-
-Once the ``WizardStep`` subclass is defined, it needs to be registered. This
-is done by calling the ``.register`` method of the ``WizardStep`` class with
-the subclass as the argument. Example::
-
-    WizardStep.register(WizardStepMetadata)
-
-This statement must be located after the subclass definition. Finally,
-the module defining the wizard step must be imported so that it is loaded
-with the rest of the code and enabled. The best place to do this is in the
-``.ready`` method of the apps' ``apps.py`` module. Example::
-
-    class TagsApp(MayanAppConfig):
-        has_rest_api = True
-        has_tests = True
-        name = 'tags'
-        verbose_name = _('Tags')
-
-        def ready(self):
-            super(TagsApp, self).ready()
-            from actstream import registry
-
-            from .wizard_steps import WizardStepTags  # NOQA
-
-The ``WizardStep`` class also allows for unregistering existing steps. This
-is accomplished by calling the ``.deregister`` method of the ``WizardStep``
-class and passing the subclass as the argument. This method should
-also be called inside the ``.ready`` method of an apps' ``apps.py``
-module. Example::
-
-
-    class TagsApp(MayanAppConfig):
-        has_rest_api = True
-        has_tests = True
-        name = 'tags'
-        verbose_name = _('Tags')
-
-        def ready(self):
-            super(TagsApp, self).ready()
-            from actstream import registry
-
-            from metadata.wizard_steps import WizardStepMetadata  # NOQA
-            from sources.wizards import WizardStep  # NOQA
-            from .wizard_steps import WizardStepTags  # NOQA
-
-            WizardStep.deregister(WizardStepTags)
-
-
-This will cause the tags assignment step to not be assigned to the upload
-wizard anymore.

docs/chapters/versioning.txt

@@ -1,6 +1,6 @@
-*******************
+===================
 Document versioning
-*******************
+===================
 
 Mayan EDMS has the ability to store different versions of the same
 document. A comment field is provided to allow users to summarize the new
@@ -46,35 +46,3 @@ There is no limit to the number of versions a document can have.
 
 By default, the last version will be showed when working with the document
 but any of the versions can be inspected and viewed.
-
-
-View a document version list
-============================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "View documents' versions list" permission is required for this action,
-    either globally or via an ACL for a document or document type.
-
-
-#. Navigate to the select document's preview view.
-#. Click on the sibebar's :guilabel:`Versions` button.
-
-
-Uploading a new document version
-================================
-
-.. admonition:: Permissions required
-    :class: warning
-
-    The "Create new document versions" permission is required for this action,
-    either globally or via an ACL for a document or document type.
-
-
-#. Navigate to the select document's versions list view.
-#. From the :guilabel:`Actions` dropdown select :guilabel:`Upload new version`.
-#. Optionally type a comment explaining the changes in the new version.
-#. Press the :guilabel:`Browse` button and select a new file.
-#. Press :guilabel:`Save` upload the new version.
-

docs/chapters/workflows.txt

@@ -1,6 +1,6 @@
-*********
+=========
 Workflows
-*********
+=========
 
 Introduction
 ============

docs/conf.py

@@ -56,7 +56,7 @@ blockdiag_latex_image_format = 'PDF'
 templates_path = ['_templates']
 
 # The suffix of source filenames.
-source_suffix = '.rst'
+source_suffix = '.txt'
 
 # The encoding of source files.
 # source_encoding = 'utf-8-sig'

docs/index.txt

@@ -17,28 +17,32 @@ repository for electronic documents.
 .. toctree::
     :hidden:
 
-    topics/features
+    parts/features
-    topics/installation
+    parts/installation
     releases/index
-    topics/getting_started
+    parts/getting_started
-    topics/adding_documents
+    parts/access_control
-    topics/access_control
+    parts/categorization
-    topics/categorization
+    parts/collaboration
-    topics/collaboration
+    parts/settings
-    topics/settings
+    parts/storage
-    topics/storage
+    chapters/rest_api
-    topics/integration
+    chapters/docker/index
-    topics/docker
+    parts/advanced
-    topics/advanced
+    parts/administration
-    topics/administration
+    parts/troubleshooting
-    topics/troubleshooting
+    parts/development
-    topics/development
+    parts/faq
-    topics/faq
+    parts/license
-    topics/license
+    parts/contact
-    topics/contact
+
 
 .. _Docker: https://www.docker.com/
 .. _Django: http://www.djangoproject.com/
 .. _Free Open Source: http://en.wikipedia.org/wiki/Open_source
 .. _Electronic Document Management System: https://en.wikipedia.org/wiki/Document_management_system
 .. _Apache 2.0 License: https://www.apache.org/licenses/LICENSE-2.0.txt
+
+
+..
+    parts/adding_documents

docs/mercs/index.txt

@@ -1,6 +1,6 @@
-*****
+=====
 MECRs
-*****
+=====
 
 Mayan EDMS Request For Comment documents index.
 

docs/parts/access_control.txt

@@ -1,6 +1,6 @@
-##############
+==============
 Access control
-##############
+==============
 
 Mayan EDMS uses a role based permission system
 (https://en.wikipedia.org/wiki/Role-based_access_control) that provides a
@@ -8,9 +8,12 @@ mechanism to control access to the contained documents and system functions.
 
 The role access control system is divided into two main groups:
 
-* Permission grants to roles for the entire system.
+- :doc:`../chapters/permissions`: Permission grants to roles for the entire system.
-* ACLs (access control lists). These are permission granted to a role for a
+- :doc:`../chapters/acls`: ACLs (access control lists). These are permission granted to a role for a
   specific object or group of objects.
 
-.. include:: ../chapters/permissions.rst
+.. toctree::
-.. include:: ../chapters/acls.rst
+    :hidden:
+
+    ../chapters/permissions
+    ../chapters/acls

docs/parts/administration.txt

@@ -0,0 +1,17 @@
+==============
+Administration
+==============
+
+- :doc:`../chapters/languages`
+- :doc:`../chapters/password_reset`
+- :doc:`../chapters/ocr_backend`
+- :doc:`../chapters/backups`
+
+
+.. toctree::
+    :hidden:
+
+    ../chapters/languages
+    ../chapters/password_reset
+    ../chapters/ocr_backend
+    ../chapters/backups

docs/parts/advanced.txt

@@ -0,0 +1,20 @@
+===============
+Advanced topics
+===============
+
+- :doc:`../chapters/workflows`
+- :doc:`../chapters/signatures`
+- :doc:`../chapters/metadata`
+- :doc:`../chapters/transformations`
+- :doc:`../chapters/versioning`
+- :doc:`../chapters/quick_labels`
+
+.. toctree::
+    :hidden:
+
+    ../chapters/workflows
+    ../chapters/signatures
+    ../chapters/metadata
+    ../chapters/transformations
+    ../chapters/versioning
+    ../chapters/quick_labels

docs/parts/categorization.txt

@@ -0,0 +1,37 @@
+==============
+Categorization
+==============
+
+Mayan EDMS several ways to visual organize documents. Document categorization
+allows showing users different views of document groups. These are divided
+into ones requiring manual maintenance and others that are updated
+automatically.
+
+
+Manual methods
+==============
+
+- :doc:`../chapters/document_types`
+- :doc:`../chapters/cabinets`
+- :doc:`../chapters/tags`
+
+.. toctree::
+  :hidden:
+
+  ../chapters/document_types
+  ../chapters/cabinets
+  ../chapters/tags
+
+
+Automatic methods
+=================
+
+- :doc:`../chapters/indexes/index`
+- :doc:`../chapters/smart_links`
+
+.. toctree::
+  :hidden:
+
+  ../chapters/indexes/index
+  ../chapters/smart_links
+

docs/parts/collaboration.txt

@@ -0,0 +1,15 @@
+=============
+Collaboration
+=============
+
+Mayan EDMS includes several methods to allow users to collectively collaborate
+when working with documents.
+
+- :doc:`../chapters/checkouts`
+- :doc:`../chapters/mailing`
+
+.. toctree::
+    :hidden:
+
+    ../chapters/checkouts
+    ../chapters/mailing

docs/parts/contact.txt

@@ -1,17 +1,16 @@
-#######
+=======
 Contact
-#######
+=======
 
 
-***
 FAQ
-***
+===
 
 * Try the :doc:`FAQ <faq>` -- it's got answers to many common questions.
 
-*****
+
 Forum
-*****
+=====
 
 Search for information or post a question, report or feature request at:
 https://forum.mayan-edms.com.
@@ -20,26 +19,9 @@ Mayan EDMS developers do their best to reply to basic questions.
 Be sure to use the search feature of the forum as some posts may already
 contain the answers to your questions.
 
-*******
-Twitter
-*******
 
-Mayan EDMS has an official Twitter account, `@mayanedms
-<https://twitter.com/mayanedms>`_, which is used for announcements and occasional
-related news.
-
-****
-Wiki
-****
-
-A community Wiki is also available for material that doesn't fit in the
-document or is being created in a collaborative manner to be added to the
-documentation. The Wiki URL is https://wiki.mayan-edms.com
-
-
-*******************
 Bugs/ticket tracker
-*******************
+===================
 
 Report bugs with Mayan EDMS or search existing ones using GitLab's `ticket tracker`_.
 

docs/parts/development.txt

@@ -0,0 +1,29 @@
+==============
+For developers
+==============
+
+- :doc:`../chapters/philosophies`
+- :doc:`../chapters/code_structure`
+- :doc:`../chapters/documentation`
+- :doc:`../chapters/source_control`
+- :doc:`../chapters/contributing`
+- :doc:`../chapters/releases`
+- :doc:`../chapters/translations`
+- :doc:`../mercs/index`
+- :doc:`../chapters/caa_entity`
+- :doc:`../chapters/caa_individual`
+
+
+.. toctree::
+    :hidden:
+
+    ../chapters/philosophies
+    ../chapters/code_structure
+    ../chapters/documentation
+    ../chapters/source_control
+    ../chapters/contributing
+    ../chapters/releases
+    ../chapters/translations
+    ../mercs/index
+    ../chapters/caa_entity
+    ../chapters/caa_individual

docs/parts/faq.txt

@@ -1,23 +1,22 @@
-###
+===
 FAQ
-###
+===
 
-*************
 Non technical
-*************
+=============
 
 Contact
-=======
+-------
 
 Can I contact members of the development team?
-----------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Direct access to members of the development team is reserved for clients
 of a support or consultation contract.
 
 
 I can't pay for support/consultation
-------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 A community forum is available at https://forum.mayan-edms.com. Members of
 the development team visit the forum on occasions and might answer some
@@ -25,13 +24,13 @@ questions on a voluntary basis.
 
 
 I found a bug
--------------
+^^^^^^^^^^^^^
 
 For bugs, open an issue at https://gitlab.com/mayan-edms/mayan-edms/issues.
 
 
 I found a security issue
-------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^
 
 Open an issue at https://gitlab.com/mayan-edms/mayan-edms/issues and mark it as
 Confidential. Allow us at least 48 hours to find and release a fix
@@ -39,7 +38,7 @@ for the issue before submitting it to the CVE database.
 
 
 I want to join the development team
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Perfect! We need all the help we can get. Tell us where we can see what you've
 done using Mayan EDMS. It could be a fork, a new UI, an API client, a custom app,
@@ -48,7 +47,7 @@ codebase.
 
 
 I submitted an issue and it has not been fixed
-----------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 There are many reason an issue might not get resolved and remain open for an
 extended period of time. We document on the issue itself the progress and when
@@ -80,7 +79,7 @@ open with no resolution are:
 
 
 I submitted a merge request and has not been merged
----------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Merge requests must follow the development standards of the code as close a
 possible. They must also be atomic and as small as possible. The code must also
@@ -89,10 +88,10 @@ customize it in some way based on your own preferences or needs.
 
 
 Distribution
-============
+------------
 
 Can I distribute Mayan EDMS under another name and/or logo?
------------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Yes, the terms of the license allow this. However, the copyright notice must
 remain intact. If you use an alternate name, a notice indicating that yours is
@@ -103,10 +102,10 @@ holders, or the core team in any way.
 
 
 Sale
-====
+----
 
 Can I sell Mayan EDMS as is or under another name?
---------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Yes, selling Mayan EDMS is permitted. However, the copyright notice must
 remain intact. If you use an alternate name, a notice indicating that yours is
@@ -119,7 +118,7 @@ type.
 
 
 Can I get exclusive distribution rights for my region?
-------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 No, the terms of the license make the project freely available to everyone.
 Restricting distribution or sale would conflict with the license terms. This
@@ -128,14 +127,14 @@ terms.
 
 
 Is there a commercial partnership program?
-------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 There was at one time but was retired. If there is interest it could be
 reinstated in the future.
 
 
 What is an EDMS?
-----------------
+^^^^^^^^^^^^^^^^
 
 EDMS stands for Electronic Document Management System and it is an more modern
 version of a DMS. A DMS is a Document Management System. A system to store,
@@ -149,15 +148,15 @@ Management). Mayan EDMS started initially as a strict EDMS project but has
 been expanding its feature set and provide some functionality from other
 system types.
 
-*********
+
 Technical
-*********
+=========
 
 Database managers
-=================
+-----------------
 
 PostgreSQL vs. MySQL
---------------------
+^^^^^^^^^^^^^^^^^^^^
 
 Since Django abstracts database operations from a functional point of view
 Mayan EDMS will behave exactly the same either way. The only concern would be
@@ -168,10 +167,10 @@ state and has to be reverted manually before trying again.
 
 
 LDAP
-====
+----
 
 How to do LDAP authentication
------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 A sample settings file called ldap_connection_settings.py is included in the
 contrib/settings/ folder of the repository showing how to setup LDAP
@@ -180,10 +179,10 @@ settings file method to use this file.
 
 
 Operating systems
-=================
+-----------------
 
 How to install Mayan EDMS in Windows operating systems?
--------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Mayan EDMS doesn't run natively on Windows. The best way is to use a virtual
 machine product, install Ubuntu or Debian, and proceed with the standard
@@ -191,10 +190,10 @@ deployment instructions or use Docker inside a GNU/Linux virtual machine.
 
 
 Python
-======
+------
 
 Is virtualenv required as specified in the documentation?
----------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 It is not necessary, but it's a strong recommendation mainly to reduce
 dependency conflicts by isolation from the main Python system install. If not
@@ -206,13 +205,13 @@ apparent reason.
 
 
 Does Mayan EDMS work with Python 3?
------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Yes, version 3.3 ships with Python 3.7 support. Support for Python 2 will
 be removed in version 4.0.
 
 How to upgrade an existing Python 2 virtualenv
-----------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Use::
 
@@ -228,10 +227,10 @@ If using a dedicated user account for Mayan EDMS::
 
 
 Static files
-============
+------------
 
 Mayan EDMS installed correctly and works, but static files are not served
--------------------------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Django's development server doesn't serve static files unless the DEBUG option
 is set to True, this mode of operation should only be used for development or
@@ -245,10 +244,10 @@ For more information check the
 
 
 Watch folders
-=============
+-------------
 
 The watched folder feature is not working
------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Make sure that the Celery BEAT scheduler is running correctly as it is the
 element that triggers the periodic tasks. Check that the user running the Mayan
@@ -257,10 +256,10 @@ EDMS services has read and write permissions for the watch folder.
 
 
 Other
-=====
+-----
 
 File system links not showing when serving content with ``Samba``
------------------------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Disable unix extensions in the [global] section and enable wide links for the file serving share
 

docs/parts/features.txt

@@ -0,0 +1,100 @@
+========
+Features
+========
+
+- :doc:`Document versioning <../chapters/versioning>`.
+
+  - Store many versions of the same document, download or revert to a
+    previous version.
+
+- :doc:`Digital signatures <../chapters/signatures>`.
+
+  - Check the authenticity of documents by verifying their embedded
+    cryptographic signatures or upload detached signatures for document
+    signed after they were stored.
+
+- :doc:`Collaboration tools <../parts/collaboration>`.
+
+  - Discuss documents, or comment on new versions of a document.
+
+- :doc:`User-defined document metadata <../chapters/metadata>`.
+
+  - Several metadata fields can be matched to a document type as per technical,
+    legal or structural requirements such as the `Dublin core`_.
+  - Metadata fields can have an initial value, which can be static or determined
+    by a template code snippet provided by the user.
+
+- :doc:`Documents can be uploaded from different sources <../chapters/sources>`.
+
+  - Local file or server side file uploads, multifunctional copier, or even via
+    email.
+
+- Batch uploads.
+
+  - Many documents can be upload in a single action.
+  - Clone a document's metadata for speedier uploads and eliminate repetitive
+    data entry.
+
+- Previews for many file formats.
+
+  - Mayan EDMS provides image preview generation for many popular file
+    formats.
+
+- Office document format support.
+
+  - Mayan EDMS can detect the presence of Libre Office and use it to support
+    word processing files, spreadsheets and presentations.
+
+- Full text searching.
+
+  - Documents can be searched by their text content, their metadata or any other
+    file attribute such as name, extension, etc.
+
+- :doc:`Configurable document grouping <../parts/categorization>`.
+
+  - Automatic linking of documents based on metadata values or document
+    properties.
+
+- :doc:`Advanced access control system <../chapters/permissions>`.
+
+  - Role based access control. It is possible to create an unlimited amount
+    of different roles not being restricted to the traditional admin,
+    operator, guest paradigm.
+  - There is a permission for every atomic operation performed by users.
+
+- Multi page document support.
+
+  - Multiple page PDF and TIFF files are supported.
+
+- :doc:`Automatic OCR processing <../chapters/ocr_backend>`.
+
+  - The task of transcribing text from documents via OCR can be distributed
+    among several physical or virtual computers to decrease load and increase
+    availability.
+  - The current language of the document is passed to the corresponding OCR
+    engine to increase the text recognition rate.
+
+- Multilingual user interface.
+
+  - Mayan EDMS being written using the Django_ framework, can be translated
+    to practically any language spoken in the world. For a list of translated
+    languages have a look at the Transifex_ project location.
+
+- :doc:`Plugable storage backends <../parts/storage>`.
+
+  - It is very easy to use 3rd party plugins such as the ones available for
+    Amazon EC2.
+
+- :doc:`Color coded tags <../chapters/tags>`.
+
+  - Labeled and color coded tags can be assigned for intuitive recognition.
+
+- :doc:`Workflows <../chapters/workflows>`.
+
+  - Keep track of the state of documents, along with the log of the previous
+    state changes.
+
+
+.. _`Dublin core`: http://dublincore.org/metadata-basics/
+.. _Django:  https://www.djangoproject.com/
+.. _Transifex: https://www.transifex.com/projects/p/mayan-edms/

docs/parts/getting_started.txt

@@ -1,13 +1,13 @@
-###############
+===============
 Getting started
-###############
+===============
 
 These are the basic concepts you need to understand to start using Mayan EDMS.
 
 The absolute minimum setup to start using Mayan EDMS is:
 
-#. A document type
+#. A document type: :doc:`../chapters/document_types`
-#. A document source to add new documents
+#. A document source to add new documents: :doc:`../chapters/sources`
 
 that's it!
 
@@ -17,6 +17,10 @@ installation. So you don't need to do anything more to start adding documents
 right now, however take your time reading the information below to get a deeper
 understanding about how these concepts work.
 
-.. include:: ../chapters/document_types.rst
+
-.. include:: ../chapters/sources.rst
+.. toctree::
+  :hidden:
+
+  ../chapters/document_types
+  ../chapters/sources
 

docs/parts/installation.txt

@@ -1,48 +1,51 @@
-############
+============
 Installation
-############
+============
 
 The easiest way to use Mayan EDMS is by using the official Docker_ image.
 Make sure Docker is properly installed and working before attempting to install
 Mayan EDMS.
 
-*****************************
 Minimum hardware requirements
-*****************************
+=============================
 
 - 2 Gigabytes of RAM (1 Gigabyte if OCR is turned off).
 - Multiple core CPU (64 bit, faster than 1 GHz recommended).
 - Unix-like operating system like GNU/Linux. For other operating systems
   user container technologies like Docker or virtual machines.
 
-****************
+
-Docker procedure
+Installation methods
-****************
+====================
+
+Docker
+------
 
 Docker is a computer program that performs operating-system-level
 virtualization also known as containerization. It allows independent
 "containers" to run within a single Linux instance, avoiding the overhead
 of starting and maintaining virtual machines (VMs).
 
-Docker can be installed using their automated script::
+- :ref:`docker_install`
 
-    wget -qO- https://get.docker.com/ | sh
+.. toctree::
+    :hidden:
 
-This installs the latest versions of Docker. If you don't want run an automated
+    ../chapters/docker/install_simple
-script follow the instructions outlined in their documentation: https://docs.docker.com/install/
 
-Once the Docker installation is finished, proceed to the link below to install
-the Docker image for Mayan EDMS.
 
-Docker image chapter: :ref:`docker_install`
-
-*******************
 Direct installation
-*******************
+-------------------
 
 For users with knowledge of Python, Django, Ubuntu, and databases.
 
-Deployments chapter: :doc:`../chapters/deploying`
+- :doc:`../chapters/deploying`
+
+
+.. toctree::
+    :hidden:
+
+    ../chapters/deploying
 
 
 .. _Docker: https://www.docker.com/

docs/parts/license.txt

@@ -1,6 +1,6 @@
-#######
+=======
 License
-#######
+=======
 
 Mayan EDMS is distributed under the `Apache 2.0 License`_.
 The complete license terms are included below.
@@ -8,8 +8,7 @@ The complete license terms are included below.
 .. _Apache 2.0 License: https://www.apache.org/licenses/LICENSE-2.0.txt
 
 
-*************
 License terms
-*************
+=============
 
 .. include:: ../../LICENSE

docs/parts/settings.txt

@@ -0,0 +1,4 @@
+.. toctree::
+    :hidden:
+
+    ../chapters/settings

docs/parts/storage.txt

@@ -0,0 +1,15 @@
+=======
+Storage
+=======
+
+Mayan EDMS stores documents in their original file format only changing the
+filename to avoid collision. By default, documents files are stored in the
+same filesystem as the program code but by using different drivers, these
+can also be placed in other storage systems like object storage.
+
+- :doc:`../chapters/file_storage`
+
+.. toctree::
+    :hidden:
+
+    ../chapters/file_storage

docs/parts/troubleshooting.txt

@@ -1,14 +1,13 @@
-###############
+===============
 Troubleshooting
-###############
+===============
 
 
-********
 Database
-********
+========
 
 MySQL error: ``OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci, IMPLICIT) and (utf8_general_ci, COERCIBLE) for operation '='”)``
-=======================================================================================================================================================
+-------------------------------------------------------------------------------------------------------------------------------------------------------
 
 ::
 
@@ -26,7 +25,7 @@ References:
 
 
 MySQL error: ``Incorrect string value: `'xE2x80x95rs6…'` for column `'content'` at row 1``
-==========================================================================================
+------------------------------------------------------------------------------------------
 
 When using MySQL and doing OCR on languages other than English
 
@@ -40,14 +39,14 @@ References:
 
 
 MySQL error: ``Error "django.db.utils.IntegrityError IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`…`.`…`, CONSTRAINT `…_refs_id_b0252274` FOREIGN KEY (`…`) REFERENCES `…` (`…`))')``
-===================================================================================================================================================================================================================================
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 
 Solution:
 Convert all MySQL tables to the same type, either all MyISAM or InnoDB
 
 
 PostgreSQL error: ``OperationalError: FATAL:  sorry, too many clients already``
-===============================================================================
+-------------------------------------------------------------------------------
 
 Set ``MAYAN_DATABASE_CONN_MAX_AGE`` to 0
 
@@ -64,12 +63,11 @@ References:
 - https://github.com/benoitc/gunicorn/issues/996
 
 
-******
 Docker
-******
+======
 
 MAYAN_APT_INSTALLS does not work for Archlinux with kernels > 4.14
-==================================================================
+------------------------------------------------------------------
 
 This is caused by a change from kernel 4.18 - 4.19. Metacopy on these kernels
 is set to yes in archlinux kernels (/sys/module/overlay/parameters/metacopy)
@@ -86,14 +84,13 @@ References:
 
 
 
-*********
 Passwords
-*********
+=========
 
 .. _troubleshooting-admin-password:
 
 Admin password reset
-====================
+--------------------
 
 To reset the password of the admin account use the following command::
 
@@ -125,7 +122,7 @@ And then execute the command::
 .. _troubleshooting-autoadmin-account:
 
 Missing automatic admin account after installation
-==================================================
+--------------------------------------------------
 
 This is caused when the ``initialsetup`` command is interrupted as the admin
 user is created outside of the database migrations.
@@ -156,12 +153,11 @@ And then execute the command::
     |MAYAN_BIN| createsuperuser
 
 
-************
 Watchfolders
-************
+============
 
 Incomplete files uploaded
-=========================
+-------------------------
 
 To avoid uploading files are they are being copied to the watchfolder, copy the
 files to a temporary directory on the same partition as the watchfolder first.
@@ -169,12 +165,12 @@ Then move the files to the watchfolder. The move will be executed as an atomic
 operation and will prevent the files to be uploaded in the middle of the
 copying process.
 
-************
+
 Dependencies
-************
+============
 
 Error: ``unable to execute 'x86_64-linux-gnu-gcc': No such file or directory``
-==============================================================================
+------------------------------------------------------------------------------
 
 This happens when using the ``MAYAN_APT_INSTALLS`` feature. It means that the
 ``GCC`` package is required to compile the packages specified with

docs/releases/0.12.txt

@@ -120,7 +120,7 @@ in the Initial data loading chapter.
 Refactored document indexing
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-:doc:`The document indexing  <../chapters/indexes>` functionality has been
+:doc:`The document indexing  <../chapters/indexes/index>` functionality has been
 improved and moved from experimental
 stage to beta stage.  Index configuration menus are now available on the
 ``Setup`` menu and allows administrators to create skeleton trees that will