matthias/mayan-edms
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'versions/minor' into clients/bc_33

Browse Source 
Signed-off-by: Roberto Rosario <roberto.rosario@mayan-edms.com>
This commit is contained in:
Roberto Rosario
2019-10-07 11:01:43 -04:00
parent 1bcc9332b2 ce4413d539
commit ce7c805251
455 changed files with 14704 additions and 7008 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/chapters/deploying.rst
View File

@@ -127,9 +127,8 @@ For another setup that offers more performance and scalability refer to the
::
sudo -u mayan MAYAN_DATABASE_ENGINE=django.db.backends.postgresql MAYAN_DATABASE_NAME=mayan \
MAYAN_DATABASE_PASSWORD=mayanuserpass MAYAN_DATABASE_USER=mayan \
MAYAN_DATABASE_HOST=127.0.0.1 MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \
sudo -u mayan MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'127.0.0.1'}}" \
MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \
/opt/mayan-edms/bin/mayan-edms.py initialsetup
@@ -148,9 +147,8 @@ For another setup that offers more performance and scalability refer to the
------------------------------------------------------------------------
::
sudo MAYAN_DATABASE_ENGINE=django.db.backends.postgresql MAYAN_DATABASE_NAME=mayan \
MAYAN_DATABASE_PASSWORD=mayanuserpass MAYAN_DATABASE_USER=mayan \
MAYAN_DATABASE_HOST=127.0.0.1 MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \
sudo mayan MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'127.0.0.1'}}" \
MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \
/opt/mayan-edms/bin/mayan-edms.py platformtemplate supervisord > /etc/supervisor/conf.d/mayan.conf
@@ -222,11 +220,11 @@ of a restart or power failure. The Gunicorn workers are increased to 3.
---------------------------------------------------------------------
Replace (paying attention to the comma at the end)::
MAYAN_BROKER_URL="redis://127.0.0.1:6379/0",
MAYAN_CELERY_BROKER_URL="redis://127.0.0.1:6379/0",
with::
MAYAN_BROKER_URL="amqp://mayan:mayanrabbitmqpassword@localhost:5672/mayan",
MAYAN_CELERY_BROKER_URL="amqp://mayan:mayanrabbitmqpassword@localhost:5672/mayan",
increase the number of Gunicorn workers to 3 in the line (``-w 2`` section)::

117
docs/chapters/docker.rst
View File

@@ -49,12 +49,7 @@ Finally create and run a Mayan EDMS container::
--name mayan-edms \
--restart=always \
-p 80:8000 \
-e MAYAN_DATABASE_ENGINE=django.db.backends.postgresql \
-e MAYAN_DATABASE_HOST=172.17.0.1 \
-e MAYAN_DATABASE_NAME=mayan \
-e MAYAN_DATABASE_PASSWORD=mayanuserpass \
-e MAYAN_DATABASE_USER=mayan \
-e MAYAN_DATABASE_CONN_MAX_AGE=0 \
-e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'172.17.0.1'}}" \
-v /docker-volumes/mayan-edms/media:/var/lib/mayan \
mayanedms/mayanedms:<version>
@@ -108,12 +103,7 @@ instead of the IP address of the Docker host (``172.17.0.1``)::
--network=mayan \
--restart=always \
-p 80:8000 \
-e MAYAN_DATABASE_ENGINE=django.db.backends.postgresql \
-e MAYAN_DATABASE_HOST=mayan-edms-postgres \
-e MAYAN_DATABASE_NAME=mayan \
-e MAYAN_DATABASE_PASSWORD=mayanuserpass \
-e MAYAN_DATABASE_USER=mayan \
-e MAYAN_DATABASE_CONN_MAX_AGE=0 \
-e MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'mayan-edms-postgres'}}" \
-v /docker-volumes/mayan-edms/media:/var/lib/mayan \
mayanedms/mayanedms:<version>
@@ -137,101 +127,14 @@ To start the container again::
Environment Variables
---------------------
The Mayan EDMS image can be configure via environment variables.
``MAYAN_DATABASE_ENGINE``
Defaults to ``None``. This environment variable configures the database
backend to use. If left unset, SQLite will be used. The database backends
supported by this Docker image are:
- ``'django.db.backends.postgresql'``
- ``'django.db.backends.mysql'``
- ``'django.db.backends.sqlite3'``
When using the SQLite backend, the database file will be saved in the Docker
volume. The SQLite database as used by Mayan EDMS is meant only for development
or testing, never use it in production.
``MAYAN_DATABASE_NAME``
Defaults to 'mayan'. This optional environment variable can be used to define
the database name that Mayan EDMS will connect to. For more information read
the pertinent Django documentation page:
:django-docs:`Connecting to the database <ref/databases/#connecting-to-the-database>`
``MAYAN_DATABASE_USER``
Defaults to 'mayan'. This optional environment variable is used to set the
username that will be used to connect to the database. For more information
read the pertinent Django documentation page:
:django-docs:`Settings, USER <ref/settings/#user>`
``MAYAN_DATABASE_PASSWORD``
Defaults to ''. This optional environment variable is used to set the
password that will be used to connect to the database. For more information
read the pertinent Django documentation page:
:django-docs:`Settings, PASSWORD <ref/settings/#password>`
``MAYAN_DATABASE_HOST``
Defaults to `None`. This optional environment variable is used to set the
hostname that will be used to connect to the database. This can be the
hostname of another container or an IP address. For more information read
the pertinent Django documentation page:
:django-docs:`Settings, HOST <ref/settings/#host>`
``MAYAN_DATABASE_PORT``
Defaults to `None`. This optional environment variable is used to set the
port number to use when connecting to the database. An empty string means
the default port. Not used with SQLite. For more information read the
pertinent Django documentation page:
:django-docs:`Settings, PORT <ref/settings/#port>`
``MAYAN_BROKER_URL``
This optional environment variable determines the broker that Celery will use
to relay task messages between the frontend code and the background workers.
For more information read the pertinent Celery Kombu documentation page: `Broker URL`_
.. _Broker URL: http://kombu.readthedocs.io/en/latest/userguide/connections.html#connection-urls
This Docker image supports using Redis and RabbitMQ as brokers.
Caveat: If the `MAYAN_BROKER_URL` and `MAYAN_CELERY_RESULT_BACKEND` environment
variables are specified, the built-in Redis server inside the container will
be disabled.
``MAYAN_CELERY_RESULT_BACKEND``
This optional environment variable determines the results backend that Celery
will use to relay result messages from the background workers to the frontend
code. For more information read the pertinent Celery Kombu documentation page:
`Task result backend settings`_
.. _Task result backend settings: http://docs.celeryproject.org/en/3.1/configuration.html#celery-result-backend
This Docker image supports using Redis and RabbitMQ as result backends.
Caveat: If the `MAYAN_BROKER_URL` and `MAYAN_CELERY_RESULT_BACKEND` environment
variables are specified, the built-in Redis server inside the container will
be disabled.
The common set of settings can also be modified via environment variables when
using the Docker image. In addition to the common set of settings, some Docker
image specific environment variables are available.
``MAYAN_SETTINGS_MODULE``
Optional. Allows loading an alternate settings file.
``MAYAN_DATABASE_CONN_MAX_AGE``
Amount in seconds to keep a database connection alive. Allow reuse of database
connections. For more information read the pertinent Django documentation
page: :django-docs:`Settings, CONN_MAX_AGE <ref/settings/#conn-max-age>`
According to new information Gunicorn's microthreads don't share connections
and will exhaust the available Postgres connections available if a number
other than 0 is used. Reference: https://serverfault.com/questions/635100/django-conn-max-age-persists-connections-but-doesnt-reuse-them-with-postgresq
and https://github.com/benoitc/gunicorn/issues/996
``MAYAN_GUNICORN_TIMEOUT``
@@ -281,6 +184,15 @@ Optional. Changes the GID of the ``mayan`` user internal to the Docker
container. Defaults to 1000.
Included drivers
----------------
The Docker image supports using Redis and RabbitMQ as result backends. For
databases, the image includes support for PostgreSQL and MySQL/MariaDB.
Support for additional brokers or databases may be added using the
``MAYAN_APT_INSTALL`` environment variable.
.. _docker-accessing-outside-data:
Accessing outside data
@@ -448,6 +360,7 @@ These are:
Nightly images
==============
The continuous integration pipeline used for testing development builds also
produces a resulting Docker image. These are build automatically and their
stability is not guaranteed. They should never be used in production.

4
docs/chapters/scaling_up.rst
View File

@@ -94,11 +94,11 @@ For the Docker image, launch a separate RabbitMQ container
docker run -d --name mayan-edms-rabbitmq -e RABBITMQ_DEFAULT_USER=mayan -e RABBITMQ_DEFAULT_PASS=mayanrabbitmqpassword -e RABBITMQ_DEFAULT_VHOST=mayan rabbitmq:3
Pass the MAYAN_BROKER_URL environment variable (https://kombu.readthedocs.io/en/latest/userguide/connections.html#connection-urls)
Pass the MAYAN_CELERY_BROKER_URL environment variable (https://kombu.readthedocs.io/en/latest/userguide/connections.html#connection-urls)
to the Mayan EDMS container so that it uses the RabbitMQ container the
message broker::
-e MAYAN_BROKER_URL="amqp://mayan:mayanrabbitmqpassword@localhost:5672/mayan",
-e MAYAN_CELERY_BROKER_URL="amqp://mayan:mayanrabbitmqpassword@localhost:5672/mayan",
When tasks finish, they leave behind a return status or the result of a
calculation, these are stored for a while so that whoever requested the

213
docs/releases/3.3.rst Normal file
View File

@@ -0,0 +1,213 @@
Version 3.3
===========
Released: XX XX, 2019
Changes
-------
- Add support for icon shadows.
- Add icons and no-result template to the object error log view and
links.
- Use Select2 widget for the document type selection form.
- Backport the vertical main menu update. This update splits the previous
main menu into a new menu in the same location as the previous one
now called the top bar, and a new vertical main menu on the left side.
The vertical menu remain open even when clicking on items and upon
a browser refresh will also restore its state to match the selected
view.
- Backport workflow preview refactor. GitLab issue #532.
- Add support for source column inheritance.
- Add support for source column exclusion.
- Backport workflow context support.
- Backport workflow transitions field support.
- Backport workflow email action.
- Backport individual index rebuild support.
- Rename the installjavascript command to installdependencies.
- Remove database conversion command.
- Remove support for quoted configuration entries. Support unquoted,
nested dictionaries in the configuration. Requires manual
update of existing config.yml files.
- Support user specified locations for the configuration file with the
CONFIGURATION_FILEPATH (MAYAN_CONFIGURATION_FILEPATH environment variable), and
CONFIGURATION_LAST_GOOD_FILEPATH
(MAYAN_CONFIGURATION_LAST_GOOD_FILEPATH environment variable) settings.
- Move bootstrapped settings code to their own module in the smart_settings apps.
- Remove individual database configuration options. All database configuration
is now done using MAYAN_DATABASES to mirror Django way of doing database setup.
- Added support for YAML encoded environment variables to the platform
templates apps.
- Move YAML code to its own module. Code now resides in common.serialization
in the form of two new functions: yaml_load and yaml_dump.
- Move Django and Celery settings. Django settings now reside in the smart
settings app. Celery settings now reside in the task manager app.
- Backport FakeStorageSubclass from versions/next. Placeholder class to allow
serializing the real storage subclass to support migrations.
Used by all configurable storages.
- Support checking in and out multiple documents.
- Remove encapsulate helper.
- Add support for menu inheritance.
- Emphasize source column labels.
- Backport file cache manager app.
- Convert document image cache to use file cache manager app.
Add setting DOCUMENTS_CACHE_MAXIMUM_SIZE defaults to 500 MB.
- Update Celery to version 4.3.0. Settings changed:
MAYAN_BROKER_URL to MAYAN_CELERY_BROKER_URL,
MAYAN_CELERY_ALWAYS_EAGER to MAYAN_CELERY_TASK_ALWAYS_EAGER.
- Replace djcelery and replace it with django-celery-beat.
- Update Celery to version 4.3.0 with 55e9b2263cbdb9b449361412fd18d8ee0a442dd3
from versions/next, code from GitLab issue #594 and GitLab merge request !55.
Thanks to Jakob Haufe (@sur5r) and Jesaja Everling (@jeverling)
for much of the research and code updates.
- Support wildcard MIME type associations for the file metadata drivers.
- Rename MAYAN_GUID to MAYAN_GID
- Update Gunicorn to use sync workers.
- Include devpi-server as a development dependency.
- Update default Docker stack file.
- Remove Redis from the Docker image.
- Add Celery flower to the Docker image.
- Allow PIP proxying to the Docker image during build.
- Default Celery worker concurrency to 0 (auto).
- Set DJANGO_SETTINGS_MODULE environment variable to make it
available to sub processes.
- Add entrypoint commands to run single workers, single gunicorn
or single celery commands like "flower".
- Add platform template to return queues for a worker.
- Remove task inspection from task manager app.
- Move pagination navigation inside the toolbar.
- Remove document image clear link and view.
This is now handled by the file caching app.
- Add web links app.
- Add support to display column help text
as a tooltip.
- Update numeric dashboard widget to display
thousand commas.
- Add support for disabling document pages.
- Add support for converter layers.
- Add redactions app.
- Unify all line endings to be Linux style.
- Add support for changing the system messages position.
GitLab issue #640. Thanks to Matthias Urhahn (@d4rken).
Removals
--------
- Database conversion. Reason for removal. The database conversions support
provided by this feature (SQLite to PostgreSQL) was being confused with
database migrations and upgrades.
Database upgrades are the responsibility of the app and the framework.
Database conversions however are not the responsibility of the app (Mayan),
they are the responsibility of the framework.
Database conversion is outside the scope of what Mayan does but we added
the code, management command, instructions and testing setup to provide
this to our users until the framework (Django) decided to add this
themselves (like they did with migrations).
Continued confusion about the purpose of the feature and confusion about
how errors with this feature were a reflexion of the code quality of
Mayannecessitated the removal of the database conversion feature.
- Django environ
Upgrading from a previous version
---------------------------------
If installed via Python's PIP
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Remove deprecated requirements::
sudo -u mayan curl https://gitlab.com/mayan-edms/mayan-edms/raw/master/removals.txt -o /tmp/removals.txt && sudo -u mayan /opt/mayan-edms/bin/pip uninstall -y -r /tmp/removals.txt
Type in the console::
/opt/mayan-edms/bin/pip install mayan-edms==3.3
the requirements will also be updated automatically.
Using Git
^^^^^^^^^
If you installed Mayan EDMS by cloning the Git repository issue the commands::
git reset --hard HEAD
git pull
otherwise download the compressed archived and uncompress it overriding the
existing installation.
Remove deprecated requirements::
pip uninstall -y -r removals.txt
Next upgrade/add the new requirements::
pip install --upgrade -r requirements.txt
Common steps
^^^^^^^^^^^^
Perform these steps after updating the code from either step above.
Make a backup of your supervisord file::
sudo cp /etc/supervisor/conf.d/mayan.conf /etc/supervisor/conf.d/mayan.conf.bck
Update the supervisord configuration file. Replace the environment
variables values show here with your respective settings. This step will refresh
the supervisord configuration file with the new queues and the latest
recommended layout::
sudo MAYAN_DATABASES="{'default':{'ENGINE':'django.db.backends.postgresql','NAME':'mayan','PASSWORD':'mayanuserpass','USER':'mayan','HOST':'127.0.0.1'}}" \
MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \
/opt/mayan-edms/bin/mayan-edms.py platformtemplate supervisord > /etc/supervisor/conf.d/mayan.conf
Edit the supervisord configuration file and update any setting the template
generator missed::
sudo vi /etc/supervisor/conf.d/mayan.conf
Migrate existing database schema with::
sudo -u mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media /opt/mayan-edms/bin/mayan-edms.py performupgrade
Add new static media::
sudo -u mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media /opt/mayan-edms/bin/mayan-edms.py preparestatic --noinput
The upgrade procedure is now complete.
Backward incompatible changes
-----------------------------
- Update quoted settings to be unquoted:
- COMMON_SHARED_STORAGE_ARGUMENTS
- CONVERTER_GRAPHICS_BACKEND_ARGUMENTS
- DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS
- DOCUMENTS_STORAGE_BACKEND_ARGUMENTS
- FILE_METADATA_DRIVERS_ARGUMENTS
- SIGNATURES_STORAGE_BACKEND_ARGUMENTS
Bugs fixed or issues closed
---------------------------
- :gitlab-issue:`526` RuntimeWarning: Never call result.get() within a task!
- :gitlab-issue:`532` Workflow preview isn't updated right after transitions are modified
- :gitlab-issue:`540` hint-outdated/update documentation
- :gitlab-issue:`594` 3.2b1: Unable to install/run under Python 3.5/3.6/3.7
- :gitlab-issue:`634` Failing docker entrypoint when using secret config
- :gitlab-issue:`635` Build a docker image for Python3
- :gitlab-issue:`640` UX: "Toast" Popup position prevents access to actions
- :gitlab-issue:`644` Update sane-utils package in docker image.
.. _PyPI: https://pypi.python.org/pypi/mayan-edms/

1
docs/releases/index.rst
View File

@@ -20,6 +20,7 @@ versions of the documentation contain the release notes for any later releases.
.. toctree::
:maxdepth: 1
3.3
3.2.8
3.2.7
3.2.6