diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ce1fd62061..95b9205ae3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -23,6 +23,7 @@ with deployments, webservers, cloud providers, etc. Code ---- +1. Complete and mail, or scan and email the corresponding Contributor Assignment Agreement: [Mayan EDMS Individual Contributor Assignment Agreement](http://mayan.readthedocs.io/en/latest/topics/caa_individual.rst) or [Mayan EDMS Entity Contributor Assignment Agreement](http://mayan.readthedocs.io/en/latest/topics/caa_entity.rst). 1. Fork [the repository](http://gitlab.com/mayan-edms/mayan-edms). 1. Choose the version for which you want to develop. The code is divided in the following branches: diff --git a/README.md b/README.md index be05663b4e..1dbea47ea4 100644 --- a/README.md +++ b/README.md @@ -49,9 +49,8 @@ The easiest way to use Mayan EDMS is by using the official [Docker](https://www.docker.com/) image. Make sure Docker is properly installed and working before attempting to install Mayan EDMS. -For the complete set of installation, configuration, upgrade, and backup -instructions visit the Mayan EDMS Docker Hub page at: -https://hub.docker.com/r/mayanedms/mayanedms/ +For the complete set of installation instructions visit the Mayan EDMS documentation +at: http://mayan.readthedocs.io/en/latest/topics/installation.html

Hardware requirements

diff --git a/README.rst b/README.rst index b9cc56dac9..7bd5d447fa 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -|pypi| |builds| |coverage| |python| |license| +|pypi| |builds| |coverage| |python| |license| |docker_pulls| |docker_stars| |docker_layers| .. image:: https://gitlab.com/mayan-edms/mayan-edms/raw/master/docs/_static/mayan_logo.png @@ -20,9 +20,8 @@ The easiest way to use Mayan EDMS is by using the official Docker_ image. Make sure Docker is properly installed and working before attempting to install Mayan EDMS. -For the complete set of installation, configuration, upgrade, and backup -instructions visit the Mayan EDMS Docker Hub page at: -https://hub.docker.com/r/mayanedms/mayanedms/ +For the complete set of installation instructions visit the Mayan EDMS documentation +at: http://mayan.readthedocs.io/en/latest/topics/installation.html .. _Docker: https://www.docker.com/ @@ -47,7 +46,6 @@ Important links - `Translations `__ - .. |pypi| image:: http://img.shields.io/pypi/v/mayan-edms.svg :target: http://badge.fury.io/py/mayan-edms .. |builds| image:: https://gitlab.com/mayan-edms/mayan-edms/badges/master/build.svg @@ -56,3 +54,9 @@ Important links :target: https://codecov.io/gitlab/mayan-edms/mayan-edms?branch=master .. |python| image:: https://img.shields.io/pypi/pyversions/mayan-edms.svg .. |license| image:: https://img.shields.io/pypi/l/mayan-edms.svg?style=flat +.. |docker_pulls| image:: https://img.shields.io/docker/pulls/mayanedms/mayanedms.svg?maxAge=3600 + :target: https://hub.docker.com/r/mayanedms/mayanedms/ +.. |docker_stars| image:: https://img.shields.io/docker/stars/mayanedms/mayanedms.svg?maxAge=3600 + :target: https://hub.docker.com/r/mayanedms/mayanedms/ +.. |docker_layers| image:: https://images.microbadger.com/badges/image/mayanedms/mayanedms.svg + :target: https://microbadger.com/images/mayanedms/mayanedms diff --git a/docs/conf.py b/docs/conf.py index f88fad3f56..6a15fcbb36 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -53,7 +53,7 @@ master_doc = 'index' # General information about the project. project = mayan.__title__ -copyright = mayan.__copyright__ +copyright = mayan.__copyright_short__ # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -179,6 +179,9 @@ html_static_path = ['_static'] # Output file base name for HTML help builder. htmlhelp_basename = 'MayanEDMSdoc' +html_show_sourcelink = False + +html_show_sphinx = False # -- Options for LaTeX output -------------------------------------------------- diff --git a/docs/index.rst b/docs/index.rst index 20e506a3ca..069919f820 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -20,10 +20,34 @@ repository for electronic documents. Installation Features Release notes and upgrading - Concepts + + ACLs + Backups + Checkouts + Document types + File storage + Document indexes + Document languages + Document signatures + Mailing + Metadata + OCR + Permissions + Settings + Sources + Smart links + Tags + Transformations + Upload wizard + Versioning + + Docker image + Advanced deployment + Development App creation - Roadmap + Pending work + Code statistics Translations Licensing FAQ diff --git a/docs/releases/3.0.rst b/docs/releases/3.0.rst index aa570fe0a2..aaf2162ee4 100644 --- a/docs/releases/3.0.rst +++ b/docs/releases/3.0.rst @@ -117,15 +117,15 @@ This will return only documents with the exact phrase “blue car”. Running multiple instances of Mayan EDMS ---------------------------------------- If you've ever tried running two instances of Mayan EDMS, you would -have noticed that they both try to create a lock file in the /tmp +have noticed that they both try to create a lock file in the ``/tmp`` directory with the same name. Only the first instance will be able to run. The lock filename needs to be unique to each instance, yet predictable so that the workers of an instance can also access the same lock file. This issues was solved by using a hexadecimal hash representation of the -installation’s unique SECRET_KEY setting. The use of a hash makes -reversing the value in order to obtain the SECRET_KEY impossible for +installation’s unique ``SECRET_KEY`` setting. The use of a hash makes +reversing the value in order to obtain the ``SECRET_KEY`` impossible for all practical purposes. @@ -275,24 +275,25 @@ Storage ------- It is now possible to pass arguments to the document, document cache and document signatures storage backends. To pass the arguments, use the new -settings: DOCUMENTS_STORAGE_BACKEND_ARGUMENTS, -DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS, and SIGNATURES_STORAGE_BACKEND_ARGUMENTS. +settings: ``DOCUMENTS_STORAGE_BACKEND_ARGUMENTS``, +``DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS``, and ``SIGNATURES_STORAGE_BACKEND_ARGUMENTS``. -The FileBasedStorage driver originally provided has been removed. -With this change the setting STORAGE_FILESTORAGE_LOCATION has also been removed. -The storage driver now default to Django's own FileSystemStorage driver. +The ``FileBasedStorage`` driver originally provided has been removed. +With this change the setting ``STORAGE_FILESTORAGE_LOCATION`` has also been removed. +The storage driver now default to Django's own ``FileSystemStorage`` driver. By using this driver each app is responsible of specifying their storage path. The path path (or location) is configure via the -DOCUMENTS_STORAGE_BACKEND_ARGUMENTS, DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS, or -SIGNATURES_STORAGE_BACKEND_ARGUMENTS for the documents, document cache and document signatures respectively. +``DOCUMENTS_STORAGE_BACKEND_ARGUMENTS``, ``DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS``, or +``SIGNATURES_STORAGE_BACKEND_ARGUMENTS`` for the documents, document cache and document signatures respectively. -For example, to change the document storage location use: +For example, to change the document storage location use:: -DOCUMENTS_STORAGE_BACKEND_ARGUMENTS = '{ location: }' -If no path is specified the backend will default to 'mayan/media/document_storage'. + DOCUMENTS_STORAGE_BACKEND_ARGUMENTS = '{ location: }' -Finally, to standardize the way app use storage, the .storages modules is now used -instead of the .runtime module. +If no path is specified the backend will default to ``mayan/media/document_storage``. + +Finally, to standardize the way app use storage, the ``storages.py`` modules is now used +instead of the ``runtime.py`` module. User event filtering -------------------- diff --git a/docs/topics/deploying.rst b/docs/topics/deploying.rst index d541cdab04..ca4cddcbda 100644 --- a/docs/topics/deploying.rst +++ b/docs/topics/deploying.rst @@ -21,258 +21,128 @@ Ubuntu If using a Debian_ or Ubuntu_ based Linux distribution, get the executable requirements using:: - apt-get install graphviz nginx supervisor redis-server postgresql \ - libpq-dev libjpeg-dev libmagic1 libpng-dev libreoffice \ - libtiff-dev gcc ghostscript gnupg python-dev python-virtualenv \ - tesseract-ocr poppler-utils -y + apt-get install g++ gcc ghostscript gnupg1 graphviz libjpeg-dev libmagic1 \ + libpq-dev libpng-dev libreoffice libtiff-dev poppler-utils postgresql \ + python-dev python-pip python-virtualenv redis-server sane-utils supervisor \ + tesseract-ocr zlib1g-dev -y -If using Ubuntu 16.10 also install GPG version 1 (as GPG version 2 is the new default for this distribution and not yet supported by Mayan EDMS) :: +Create an user account for the installation:: - apt-get install gnupg1 -y + sudo adduser mayan --disabled-password --disabled-login --no-create-home --gecos "" +Create the parent directory where the project will be deployed:: -Mac OSX -------- - -Mayan EDMS is dependent on a number of binary packages and the recommended -way is to use a package manager such as `MacPorts `_ -or `Homebrew `_. - - -Use MacPorts to install binary dependencies -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -With MacPorts installed run the command: - -.. code-block:: bash - - sudo port install python-dev gcc tesseract-ocr unpaper \ - python-virtualenv ghostscript libjpeg-dev libpng-dev \ - poppler-utils - -Set the Binary paths -******************** - -Mayan EDMS by default will look in /usr/bin/ for the binary files it needs -so either you can symlink the binaries installed via MacPorts in /opt/local/bin/ -to /usr/bin/ with ... - -.. code-block:: bash - - sudo ln -s /opt/local/bin/tesseract /usr/bin/tesseract - -Alternatively, set the paths in the ``settings/locals.py`` - -.. code-block:: python - - LIBREOFFICE_PATH = '/Applications/LibreOffice.app/Contents/MacOS/soffice' - -Or Use Homebrew -~~~~~~~~~~~~~~~ - -With Homebrew installed run the command: - -.. code-block:: bash - - brew install python gcc tesseract unpaper poppler libpng postgresql - -Set the Binary paths -******************** - -Mayan EDMS by default will look in /usr/bin/ for the binary files it needs. -You can symlink the binaries installed via brew in /usr/local/bin/ -to /usr/bin/ with: - -.. code-block:: bash - - sudo ln -s /usr/local/bin/tesseract /usr/bin/tesseract && \ - sudo ln -s /usr/local/bin/unpaper /usr/bin/unpaper && \ - sudo ln -s /usr/local/bin/pdftotext /usr/bin/pdftotext && \ - sudo ln -s /usr/local/bin/gs /usr/bin/gs - -Alternatively, set the paths in the ``settings/locals.py`` - -.. code-block:: python - - LIBREOFFICE_PATH = '/Applications/LibreOffice.app/Contents/MacOS/soffice' - - -Common steps ------------- -Switch to superuser:: - - sudo -i - -Change to the directory where the project will be deployed:: - - cd /usr/share + sudo mkdir -p /opt Create the Python virtual environment for the installation:: - virtualenv mayan-edms + sudo virtualenv /opt/mayan-edms -Activate the virtualenv:: +Create the folder for the Mayan EDMS data:: - source mayan-edms/bin/activate + sudo mkdir /opt/mayan-edms/media + +Make the mayan user the owner of the installation directory:: + + sudo chown mayan:mayan /opt/mayan-edms -R Install Mayan EDMS from PyPI:: - pip install mayan-edms + sudo -u mayan /opt/mayan-edms/bin/pip --no-cache-dir install mayan-edms -Install the Python client for PostgreSQL, Redis, and uWSGI:: +Install the Python client for PostgreSQL and Redis:: - pip install psycopg2 redis uwsgi + sudo -u mayan /opt/mayan-edms/bin/pip install --no-cache-dir psycopg2==2.7.3.2 redis==2.10.6 Create the database for the installation:: - sudo -u postgres createuser -P mayan (provide password) + sudo -u postgres psql -c "CREATE USER mayan WITH password 'mayanuserpass';" sudo -u postgres createdb -O mayan mayan -Create the directory for the log files:: +Initialize the project:: - mkdir /var/log/mayan - -Change the current directory to be the one of the installation:: - - cd mayan-edms - -Make a convenience symbolic link:: - - ln -s lib/python2.7/site-packages/mayan . - -Create an initial settings file:: - - mayan-edms.py createsettings - -Append the following to the ``mayan/settings/local.py`` file, paying attention to replace the ``PASSWORD`` value:: - - DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.postgresql_psycopg2', - 'NAME': 'mayan', - 'USER': 'mayan', - 'PASSWORD': '', - 'HOST': 'localhost', - 'PORT': '5432', - } - } - - BROKER_URL = 'redis://127.0.0.1:6379/0' - CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379/0' - -If using Ubuntu 16.10, also add this line to the ``mayan/settings/local.py`` file:: - - SIGNATURES_GPG_PATH = '/usr/bin/gpg1' - -Migrate the database or initialize the project:: - - mayan-edms.py initialsetup - -Disable the default NGINX site:: - - rm /etc/nginx/sites-enabled/default - -Create a ``uwsgi.ini`` file with the following contents:: - - [uwsgi] - chdir = /usr/share/mayan-edms/lib/python2.7/site-packages/mayan - chmod-socket = 664 - chown-socket = www-data:www-data - env = DJANGO_SETTINGS_MODULE=mayan.settings.production - gid = www-data - logto = /var/log/uwsgi/%n.log - pythonpath = /usr/share/mayan-edms/lib/python2.7/site-packages - master = True - max-requests = 5000 - socket = /usr/share/mayan-edms/uwsgi.sock - uid = www-data - vacuum = True - wsgi-file = /usr/share/mayan-edms/lib/python2.7/site-packages/mayan/wsgi.py - -Create the directory for the uWSGI log files:: - - mkdir /var/log/uwsgi - -Create the NGINX site file for Mayan EDMS, ``/etc/nginx/sites-available/mayan``:: - - server { - listen 80; - server_name localhost; - - location / { - include uwsgi_params; - uwsgi_pass unix:/usr/share/mayan-edms/uwsgi.sock; - - client_max_body_size 30M; # Increse if your plan to upload bigger documents - proxy_read_timeout 30s; # Increase if your document uploads take more than 30 seconds - } - - location /static { - alias /usr/share/mayan-edms/mayan/media/static; - expires 1h; - } - - location /favicon.ico { - alias /usr/share/mayan-edms/mayan/media/static/appearance/images/favicon.ico; - expires 1h; - } - } - -Enable the NGINX site for Mayan EDMS:: - - ln -s /etc/nginx/sites-available/mayan /etc/nginx/sites-enabled/ - -Create the supervisor file for the uWSGI process, ``/etc/supervisor/conf.d/mayan-uwsgi.conf``:: - - [program:mayan-uwsgi] - command = /usr/share/mayan-edms/bin/uwsgi --ini /usr/share/mayan-edms/uwsgi.ini - user = root - autostart = true - autorestart = true - redirect_stderr = true - -Create the supervisor file for the Celery worker, ``/etc/supervisor/conf.d/mayan-celery.conf``:: - - [program:mayan-worker] - command = /usr/share/mayan-edms/bin/python /usr/share/mayan-edms/bin/mayan-edms.py celery --settings=mayan.settings.production worker -Ofair -l ERROR - directory = /usr/share/mayan-edms - user = www-data - stdout_logfile = /var/log/mayan/worker-stdout.log - stderr_logfile = /var/log/mayan/worker-stderr.log - autostart = true - autorestart = true - startsecs = 10 - stopwaitsecs = 10 - killasgroup = true - priority = 998 - - [program:mayan-beat] - command = /usr/share/mayan-edms/bin/python /usr/share/mayan-edms/bin/mayan-edms.py celery --settings=mayan.settings.production beat -l ERROR - directory = /usr/share/mayan-edms - user = www-data - numprocs = 1 - stdout_logfile = /var/log/mayan/beat-stdout.log - stderr_logfile = /var/log/mayan/beat-stderr.log - autostart = true - autorestart = true - startsecs = 10 - stopwaitsecs = 1 - killasgroup = true - priority = 998 + sudo -u mayan MAYAN_DATABASE_ENGINE=django.db.backends.postgresql MAYAN_DATABASE_NAME=mayan \ + MAYAN_DATABASE_PASSWORD=mayanuserpass MAYAN_DATABASE_USER=mayan \ + MAYAN_DATABASE_HOST=127.0.0.1 MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \ + /opt/mayan-edms/bin/mayan-edms.py initialsetup Collect the static files:: - mayan-edms.py collectstatic --noinput + sudo -u mayan MAYAN_MEDIA_ROOT=/opt/mayan-edms/media \ + /opt/mayan-edms/bin/mayan-edms.py collectstatic --noinput -Make the installation directory readable and writable by the webserver user:: +Create the supervisor file at ``/etc/supervisor/conf.d/mayan.conf``:: - chown www-data:www-data /usr/share/mayan-edms -R + [supervisord] + environment= + MAYAN_ALLOWED_HOSTS="*", # Allow access to other network hosts other than localhost + MAYAN_CELERY_RESULT_BACKEND="redis://127.0.0.1:6379/0", + MAYAN_BROKER_URL="redis://127.0.0.1:6379/0", + PYTHONPATH=/opt/mayan-edms/lib/python2.7/site-packages:/opt/mayan-edms/data, + MAYAN_MEDIA_ROOT=/opt/mayan-edms/media, + MAYAN_DATABASE_ENGINE=django.db.backends.postgresql, + MAYAN_DATABASE_HOST=127.0.0.1, + MAYAN_DATABASE_NAME=mayan, + MAYAN_DATABASE_PASSWORD=mayanuserpass, + MAYAN_DATABASE_USER=mayan, + MAYAN_DATABASE_CONN_MAX_AGE=60, + MAYAN_SIGNATURES_GPG_PATH=/usr/bin/gpg1 + DJANGO_SETTINGS_MODULE=mayan.settings.production + + [program:mayan-gunicorn] + autorestart = true + autostart = true + command = /opt/mayan-edms/bin/gunicorn -w 2 mayan.wsgi --max-requests 500 --max-requests-jitter 50 --worker-class gevent --bind 0.0.0.0:8000 + user = mayan + + [program:mayan-worker-fast] + autorestart = true + autostart = true + command = nice -n 1 /opt/mayan-edms/bin/mayan-edms.py celery worker -Ofair -l ERROR -Q converter -n mayan-worker-fast.%%h --concurrency=1 + killasgroup = true + numprocs = 1 + priority = 998 + startsecs = 10 + stopwaitsecs = 1 + user = mayan + + [program:mayan-worker-medium] + autorestart = true + autostart = true + command = nice -n 18 /opt/mayan-edms/bin/mayan-edms.py celery worker -Ofair -l ERROR -Q checkouts_periodic,documents_periodic,indexing,metadata,sources,sources_periodic,uploads,documents -n mayan-worker-medium.%%h --concurrency=1 + killasgroup = true + numprocs = 1 + priority = 998 + startsecs = 10 + stopwaitsecs = 1 + user = mayan + + [program:mayan-worker-slow] + autorestart = true + autostart = true + command = nice -n 19 /opt/mayan-edms/bin/mayan-edms.py celery worker -Ofair -l ERROR -Q mailing,tools,statistics,parsing,ocr -n mayan-worker-slow.%%h --concurrency=1 + killasgroup = true + numprocs = 1 + priority = 998 + startsecs = 10 + stopwaitsecs = 1 + user = mayan + + [program:mayan-celery-beat] + autorestart = true + autostart = true + command = nice -n 1 /opt/mayan-edms/bin/mayan-edms.py celery beat --pidfile= -l ERROR + killasgroup = true + numprocs = 1 + priority = 998 + startsecs = 10 + stopwaitsecs = 1 + user = mayan Enable and restart the services [1_]:: systemctl enable supervisor systemctl restart supervisor - systemctl restart nginx [1]: https://bugs.launchpad.net/ubuntu/+source/supervisor/+bug/1594740 diff --git a/docs/topics/development.rst b/docs/topics/development.rst index 0bddb6ec3d..4dfeee6f85 100644 --- a/docs/topics/development.rst +++ b/docs/topics/development.rst @@ -253,12 +253,7 @@ Steps to deploy a development version Contributing changes -------------------- -Once your have created and committed some new code or feature, submit a Pull -Request. Be sure to merge with the development branch before doing a Pull -Request so that patches apply as cleanly as possible. If there are no conflicts, -Merge Requests can be merged directly from the website UI otherwise a manual -command line merge has to be done and your patches might take longer to get -merged. +Follow the latest contributing guidelines outlined here: https://gitlab.com/mayan-edms/mayan-edms/blob/master/CONTRIBUTING.md Debugging @@ -331,7 +326,7 @@ first need to install the documentation editing dependencies with:: Then, to build an HTML version of the documentation, run the following command from the **docs** directory:: - $ make docs_serve + $ make docs-serve The generated documentation can be viewed by browsing to http://127.0.0.1:8000 or by browsing to the ``docs/_build/html`` directory. @@ -404,50 +399,52 @@ Release checklist 1. Check for missing migrations:: - $ make check_missing_migrations + make check-missing-migrations 2. Synchronize translations:: - $ make translations_pull + make translations-pull 3. Compile translations:: - $ make translations_compile + make translations-compile 4. Write release notes. 5. Update changelog. -6. Update requirements version in `setup.py` -7. Bump version in `mayan/__init__.py` -8. Scan the code with flake8 for simple style warnings. -9. Check README.rst format with:: +6. Scan the code with flake8 for simple style warnings. +7. Check README.rst format with:: - $ python setup.py check -r -s + python setup.py check -r -s or with:: - $ make check_readme + make check-readme +8. Bump version in `mayan/__init__.py` and in `docker/version`. +9. Update requirements version in `setup.py` using:: + + make generate-setup 10. Build source package and test:: - $ make test_sdist_via_docker_ubuntu + make test-sdist-via-docker-ubuntu 11. Build wheel package and test:: - $ make test_whell_via_docker_ubuntu + make test-wheel-via-docker-ubuntu 12. Tag version:: - $ git tag -a vX.Y.Z -m "Version X.Y.Z" + git tag -a vX.Y.Z -m "Version X.Y.Z" 13. Push tag upstream:: - $ git push --tags + git push --tags 14. Build and upload a test release:: - $ make release_test_via_docker_ubuntu + make release-test-via-docker-ubuntu 15. Build and upload a final release:: - $ make release_via_docker_ubuntu + make release-via-docker-ubuntu diff --git a/docs/topics/docker.rst b/docs/topics/docker.rst index 98b1c5a04a..0c5d3f8134 100644 --- a/docs/topics/docker.rst +++ b/docs/topics/docker.rst @@ -127,6 +127,25 @@ connections. For more information read the pertinent Django documentation page: [Settings, CONN_MAX_AGE](https://docs.djangoproject.com/en/1.10/ref/settings/#conn-max-age) +``MAYAN_SETTINGS_FILE`` + +Optional. Previously only the local.py file was the only settings file +available to allow users to make configuration changes to their installations. +Now with this environment variable, users are free to create multiple settings +files and tell the Mayan EDMS container which setting file to import. The +only requirement is that the setting file starts with a global import of +mayan.settings.production. In the form:: + + from mayan.settings.production import * + + +``MAYAN_GUNICORN_WORKERS`` + +Optional. This environment variable controls the number of frontend workers +that will be executed. If not specified the default is 2. For heavier loads, +user a higher number. A formula recommended for this setting is the number +of CPU cores + 1. + Accessing outside data ====================== diff --git a/docs/topics/faq.rst b/docs/topics/faq.rst index 987e0d2a43..913330eba5 100644 --- a/docs/topics/faq.rst +++ b/docs/topics/faq.rst @@ -121,6 +121,8 @@ A the moment no, but it is something being considered. Not at the moment. When all the projects and libraries upon which Mayan is dependent support Python 3 then will the project move to fully support Python 3. +Update 2018-06-04: Python 3 will be required from Django 2.0 onwards. As such +this is now a bigger priority and the transition to Python 3 has begun. **Q: The watched folder feature is not working** diff --git a/docs/topics/index.rst b/docs/topics/index.rst index 1365232361..c937e20d94 100644 --- a/docs/topics/index.rst +++ b/docs/topics/index.rst @@ -1,5 +1,5 @@ -Concepts -======== +Topics +====== Introductions to all the key parts of Mayan EDMS you'll need to know: diff --git a/docs/topics/installation.rst b/docs/topics/installation.rst index 5b86791a60..37bf9cd6a0 100644 --- a/docs/topics/installation.rst +++ b/docs/topics/installation.rst @@ -19,14 +19,14 @@ virtualization also known as containerization. It allows independent "containers" to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines (VMs). -:ref:`docker` +Docker image chapter: :ref:`docker` -Advanced deployment +Direct installation ------------------- For users with knowledge of Python, Django, Ubuntu, and databases. -:ref:`deploying` +Advanced deployment chapter: :ref:`deploying` .. _Docker: https://www.docker.com/ diff --git a/docs/topics/pending_work.rst b/docs/topics/pending_work.rst index ebaeb0818c..e47d81ebcd 100644 --- a/docs/topics/pending_work.rst +++ b/docs/topics/pending_work.rst @@ -46,60 +46,6 @@ Forms processing - Remove usage of self.cleaned_data. Use self.clean_data instead. -Other -~~~~~ -- Python based Javascript package manager. Each app specifies what - library and version needs. The common app (or a new app) will add all - the JS loading lines automatically so that compress can detect them. -- When moving documents to the trash update the message to "submitted" - and not "moved" or "deleted" since this is handled by a task queue - and is not immediate and doesn't delete the document. -- When emptying the trash update the message to "submitted" - since this is handled by a task queue and is not immediate. -- New app that allows creating user document filters. Will provide the - same service as the document filters class. Interface can be made - using the template language or the same UI as the smart links. -- Allow add queue metadata that can be exported via a management command. - This will allow creating supervisor templates without all the worker - entries being hardcoded. -- Delete .gitignore files from copied packages. Include .gitignore files - keep compiled or distributable files from being included in the main - repository. Temporary measure until a Javascript library manager is - added. -- Automatically capture license information from installed Javascript - libraries. -- Automatically capture license information from installed Python - packages. -- Update celery to 4.1.0 -- Update to use the new class based views in the authentication app. - password_change(), password_change_done(), password_reset(), - password_reset_done(), password_reset_confirm(), and password_reset_complete() - function-based views are deprecated in favor of new class-based views - PasswordChangeView, PasswordChangeDoneView, PasswordResetView, - PasswordResetDoneView, PasswordResetConfirmView, and PasswordResetCompleteView. -- django.utils.translation.string_concat() is deprecated in favor of - django.utils.text.format_lazy(). string_concat(strings) can be - replaced by format_lazy('{}' x len(strings), strings). - Found in converter/classes.py and metadata/forms.py. -- Fix warnings in preparation for Django 2.0. -- Update all tempfile.mkstemp() to tempfile.mkstemp(dir=setting_temporary_directory.value) -- Get rid of common.utils.get_descriptor only used by common.utils.copyfile -- Update common.utils.copyfile to use only file objects. -- Change metadata label column from CharField to Label -- Start testing to Python 3 compatibility. -- Unify all RelationshipForms into a common class. -- Add test for event subscription view. -- Repeated templates: password_reset_confirm.html and password_reset_form.html -- Remove unused text=get_notification_count from events.links -- Reduce number of languages so dropzone view starts faster. -- In DocumentComment model, rename comment field to text. -- class MultipleInstanceActionMixin is deprecated. Replace views using this with - MultipleObjectFormActionView or MultipleObjectConfirmActionView -- Subscribe to tag events -- Subscribe to workflow events -- Subscribe to workflow state events -- Link to go from event notification to view. Example document workflow notification to document workflow views. - Permissions ~~~~~~~~~~~ - Permission should be reciprocal. Example: To be able to add a tag to a @@ -228,37 +174,6 @@ Distribution the JS loading lines automatically so that compress can detect them. -Other -~~~~~ -- New app that allows creating user document filters. Will provide the - same service as the document filters class. Interface can be made - using the template language or the same UI as the smart links. -- Allow add queue metadata that can be exported via a management command. - This will allow creating supervisor templates without all the worker - entries being hardcoded. -- Automatically capture license information from installed Javascript - libraries. -- Automatically capture license information from installed Python - packages. -- Finish and merge improved compressed file branch. -- Improve and merge PCL support branch. -- Swtich to self hosted documentation. -- Unify error logs in a common model. Fields: Datetime, namespace, - message, content type, object id. -- Export documents as PDF. Each document image is used to create a PDF - dinamycally. -- Document splitting. Only for PDF files first. A document versions - relationship between the documents has to be designed. -- Manually linking documents. -- Migrate settings/base.py to Django's 1.11 format. -- Rename model methods to use ``get_`` or ``do_`` -- Hunt TODO -- Hunt FIXME -- Convert SETTING_GPG_BACKEND into a setting option similar to converter and converter options. -- Reorganize modelForms Meta class and methods. -- Periodic messages, like notifications. Using an AJAX worker. - - Metadata ~~~~~~~~ - Metadata lookup memory. Add a select2 style widget that will query a @@ -296,3 +211,102 @@ Workflows UI as the smart links app. Will allow restricting the firing of workflow actions by an user defined filter criteria. - New workflow action: send email. Subject and content are templates. + + +Other +~~~~~ +- Python based Javascript package manager. Each app specifies what + library and version needs. The common app (or a new app) will add all + the JS loading lines automatically so that compress can detect them. +- When moving documents to the trash update the message to "submitted" + and not "moved" or "deleted" since this is handled by a task queue + and is not immediate and doesn't delete the document. +- When emptying the trash update the message to "submitted" + since this is handled by a task queue and is not immediate. +- New app that allows creating user document filters. Will provide the + same service as the document filters class. Interface can be made + using the template language or the same UI as the smart links. +- Allow add queue metadata that can be exported via a management command. + This will allow creating supervisor templates without all the worker + entries being hardcoded. +- Delete .gitignore files from copied packages. Include .gitignore files + keep compiled or distributable files from being included in the main + repository. Temporary measure until a Javascript library manager is + added. +- Automatically capture license information from installed Javascript + libraries. +- Automatically capture license information from installed Python + packages. +- Update celery to 4.1.0 +- Update to use the new class based views in the authentication app. + password_change(), password_change_done(), password_reset(), + password_reset_done(), password_reset_confirm(), and password_reset_complete() + function-based views are deprecated in favor of new class-based views + PasswordChangeView, PasswordChangeDoneView, PasswordResetView, + PasswordResetDoneView, PasswordResetConfirmView, and PasswordResetCompleteView. +- django.utils.translation.string_concat() is deprecated in favor of + django.utils.text.format_lazy(). string_concat(strings) can be + replaced by format_lazy('{}' x len(strings), strings). + Found in converter/classes.py and metadata/forms.py. +- Fix warnings in preparation for Django 2.0. +- Update all tempfile.mkstemp() to tempfile.mkstemp(dir=setting_temporary_directory.value) +- Get rid of common.utils.get_descriptor only used by common.utils.copyfile +- Update common.utils.copyfile to use only file objects. +- Change metadata label column from CharField to Label +- Start testing to Python 3 compatibility. +- Unify all RelationshipForms into a common class. +- Add test for event subscription view. +- Repeated templates: password_reset_confirm.html and password_reset_form.html +- Remove unused text=get_notification_count from events.links +- Reduce number of languages so dropzone view starts faster. +- In DocumentComment model, rename comment field to text. +- class MultipleInstanceActionMixin is deprecated. Replace views using this with + MultipleObjectFormActionView or MultipleObjectConfirmActionView +- Subscribe to tag events +- Subscribe to workflow events +- Subscribe to workflow state events +- Link to go from event notification to view. Example document workflow notification to document workflow views. +- Find replacement for ``python-gnupg``. Unstable & inconsistent API. +- New app that allows creating user document filters. Will provide the + same service as the document filters class. Interface can be made + using the template language or the same UI as the smart links. +- Allow add queue metadata that can be exported via a management command. + This will allow creating supervisor templates without all the worker + entries being hardcoded. +- Automatically capture license information from installed Javascript + libraries. +- Automatically capture license information from installed Python + packages. +- Finish and merge improved compressed file branch. +- Improve and merge PCL support branch. +- Swtich to self hosted documentation. +- Unify error logs in a common model. Fields: Datetime, namespace, + message, content type, object id. +- Export documents as PDF. Each document image is used to create a PDF + dinamycally. +- Document splitting. Only for PDF files first. A document versions + relationship between the documents has to be designed. +- Manually linking documents. +- Migrate settings/base.py to Django's 1.11 format. +- Rename model methods to use ``get_`` or ``do_`` +- Hunt TODO +- Hunt FIXME +- Convert SETTING_GPG_BACKEND into a setting option similar to converter and converter options. +- Reorganize modelForms Meta class and methods. +- Periodic messages, like notifications. Using an AJAX worker. +- Find replacement for ``python-gnupg``. Unstable & inconsistent API. +- Google docs integration. Upload document from Google Drive. +- Get ``dumpdata`` and ``loaddata`` working flawlessly. Will allow for easier backups, restores and database backend migrations. +- Add generic list ordering. ``django.views.generic.list.MultipleObjectMixin`` (https://docs.djangoproject.com/en/1.8/ref/class-based-views/mixins-multiple-object/#django.views.generic.list.MultipleObjectMixin) now supports an ``ordering`` parameter. +- Add support to convert any document to PDF. https://gitlab.mister-muffin.de/josch/img2pdf +- Add support for combining documents. +- Add support for splitting documents. +- Add new document source to get documents from an URL. +- Add support for metadata mapping files. CSV file containing filename to metadata values mapping, useful for bulk upload and migrations. +- Add support for registering widgets to the home screen. +- Merge mimetype and converter apps. +- Metadata widgets (Date, time, timedate). +- Datatime widget: https://github.com/smalot/bootstrap-datetimepicker +- Add events for document signing app (uploaded detached signateure, signed document, deleted signature) +- A configurable conversion process. Being able to invoke different binaries for file conversion, as opposed to the current libreoffice only solution. +- A tool in the admin interface to mass (re)convert the files (basically the page count function, but then applied on all documents). diff --git a/docs/topics/settings.rst b/docs/topics/settings.rst index 17e1e89c97..c6302ea7a3 100644 --- a/docs/topics/settings.rst +++ b/docs/topics/settings.rst @@ -2,17 +2,77 @@ Settings ======== -When Mayan EDMS is initially installed a ``local.py`` file is created inside the -``/mayan/settings/`` folder. So if you installed Mayan EDMS according to the -instructions provided in this documentation your ``local.py`` should be located in -the directory: ``/usr/share/mayan-edms/mayan/settings/local.py``. +Mayan EDMS can be configure via environment variables or by setting files. -For a list of all the configuration options, go to "Setup" then "Settings" on -your browser. This is also a good place to check if your overrided setting -option value in your ``local.py`` file is being interpreted correctly. +Via environment variables +========================= -Settings can also be changed via environment variables by prepending the string -``"MAYAN_"`` to the configuration name. For example, to change the number of documents -displayed per page (COMMON_PAGINATE_BY, by default 40), use:: +To use environment variables, lookup the name of the setting you want to +override in the "Settings" menu. The "Settings" menu is located inside the +"Setup" main menu. To pass a value via an environment variable append +``"MAYAN_"`` to the name of the settings option. For example, to change +the number of documents displayed per page (COMMON_PAGINATE_BY, by default 40), +use:: - MAYAN_COMMON_PAGINATE_BY=10 + export MAYAN_COMMON_PAGINATE_BY=10 + +Restart Mayan EDMS and the new value will take effect. The "Settings" menu +can be used to verify if the overrided setting value is being interpreted +correctly. + +Via settings files +================== + +Another way to configure Mayan EDMS is via Python-style, settings files. +If Mayan EDMS was installed using the Python package a ``mayan_settings`` +folder will created for this purpose. If you installed Mayan EDMS +according to the :ref:`deploying` instructions provided in this +documentation your ``mayan_settings`` folder should be located in the directory: +``/usr/share/mayan-edms/mayan/media/mayan_settings``. + +If Mayan EDMS was installed using Docker, the ``mayan_settings`` folder +will be found inside the install Docker volume. If you installed Mayan EDMS +according to the :ref:`docker` instructions provided in this documentation +your ``mayan_settings`` folder should be located in the directory: +``/docker-volumes/mayan/mayan_settings``. + +Create a file with any valid name and a ``.py`` extension in the +``mayan_settings`` folder. The file must starts with a global import of +``mayan.settings.production``. In the form:: + + from mayan.settings.production import * + +Now add the corresponding lines to override the default settings. +In the settings file, it is not necesary to prepend the string ``MAYAN_`` to +the setting option. For example, to change the number of documents displayed +per page (COMMON_PAGINATE_BY, by default 40), +use:: + + COMMON_PAGINATE_BY=10 + +versus:: + + export MAYAN_COMMON_PAGINATE_BY=10 + +when using the environment variable method. + +For this example let's assume the file was saved with the name ``mysettings.py``. + +The way used to tell Mayan EDMS to import this file will vary based on the +installation method. + +For the :ref:`deploying` method, the full import path will be +``mayan.media.mayan_settings.mysettings`` and can be passed via the +``--settings`` command line argument like this:: + + python manage.py runserver --settings=mayan.media.mayan_settings.mysettings + +or via the ``DJANGO_SETTINGS_MODULE`` environment variable like this:: + + export DJANGO_SETTINGS_MODULE=mayan.media.mayan_settings.mysettings + +For the :ref:`docker` installation method, the full import path will be +``mayan_settings.mysettings`` and can only be passed via the +``MAYAN_SETTINGS_MODULE`` environment variable like this:: + + docker run <...> -e MAYAN_SETTINGS_MODULE=mayan_settings.mysettings