Commit Graph

6850 Commits

Author SHA1 Message Date
Roberto Rosario
cfe1934b9b Appearance: Fix form CSS media rendering
Fix the way the form CSS contained in the media attribute
is rendered. This is now an interator and not a single value.
Replace the current method with a for loop.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-29 01:12:36 -04:00
Roberto Rosario
ef3453b48c Documentation: Fix error in example setting
The example MAYAN_DATABASES setting was missing the 'default'
database key.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 15:15:39 -04:00
Roberto Rosario
2bd649ab52 Documentation: Add install troubleshooting
Add section to outline common pitfalls when installing.
Reference GitLab issue #487.

Update installation instructons to use the setting
MAYAN_DATABASES instead of the old MAYAN_DATABASE_*.

Unify the installation instructions by converting the
chapters into partials that are now included in the
topic file.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 05:09:49 -04:00
Roberto Rosario
13524f5ce0 Documentation: Update watch folder description
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 04:14:33 -04:00
Roberto Rosario
c6104e0080 Mirroring: Increase cache timeouts
Increase the default value of the index mirroring cache timeout
from 10 seconds to 5 minutes. Since version 3.1.5, mirror
cache invalidation is tied to index updates. This makes the
timeout less relevant. The purpose of the cache timeout is
now avoid runaway memory usage.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 04:13:48 -04:00
Roberto Rosario
f9a0fb2e79 Watch folders: Add support for subfolders
Add support for subfolder scanning to watch folders. Closes
GitLab issue #498.

This commit adds a new field to watch folders called
"include_subdirectories".

The directory walk was also updated to use pathlib2.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 04:00:15 -04:00
Roberto Rosario
d973a20c30 Mailing: Add support for a from field
Add support to the mailing profiles for specifying a "from"
address. Closes GitLab issue #522.

This commit adds a new backend class property "class_fields"
which differs from the normal "fields" property. The "class_fields"
property specifies which of the backend fields will be used to
initialize a backend's driver class. This is to avoid passing
fields that the driver doesn't expect and getting an error.

When sending emails, the "send" method will attempt to get
a "from" key from the backend data and use that when sending
emails. If no "from" key is found a None is passes. Django's
behavior in this situation dictates that the "from" value will
then be taken from the DEFAULT_FROM_EMAIL setting.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 02:59:12 -04:00
Roberto Rosario
b8b10592c7 Mailing: Display a message when testing
A success or failure message will be now displayed when
testing a mailing profile.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 02:57:23 -04:00
Roberto Rosario
96ee283226 Documentation: Improve app chapter formatting
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 00:36:42 -04:00
Roberto Rosario
0188737e74 Settings: Delay initialization of setting
Delay the usage and initialization of Django settings
by removing their reference from the "save_configuration"
method declaration.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 00:35:05 -04:00
Roberto Rosario
46bd5b0a17 Documentation: Improve versioning section
Fix formatting of the versioning section
and turn the list of version examples into a table.
Add further examples for minor and micro releases.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-28 00:33:42 -04:00
Roberto Rosario
10ccab662f Password validations: Add two new validators
Add two new custom password validators. One ensures
passwords have a minimum number of uppercase letters and the
other ensures passwords have a minimum amount of numbers.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 20:07:18 -04:00
Roberto Rosario
77dc53f244 Documentation: Add mention to DEFAULT_FROM_EMAIL
Add mention and example of this new settings option
that is now available.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 20:05:27 -04:00
Roberto Rosario
21d7ec2428 Documentation: Cleanup and code documentation
Cleanup the Sphinx configuration file to comply with PEP8.

Enable the viewcode and autodoc extensions to start adding
code snippets in code.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 20:04:50 -04:00
Roberto Rosario
2ca38c20b0 Tests: Fix failing tests
Fix failing tests in the OCR and parsing apps.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 17:17:07 -04:00
Roberto Rosario
67e79d0e19 OCR, Parsing: Revert iterator stop
Revert how the OCR and document parsing generators end
their iteration. Originally they issue an empty return,
then a blank yield was added. This commit reverts the
blank yield and restores the original 'return' behavior.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 17:15:38 -04:00
Roberto Rosario
141d79afa6 Documentation: Add initial 3.2 release notes file
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:31:18 -04:00
Roberto Rosario
e9411514c7 PEP8: Code cleanup
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:28:55 -04:00
Roberto Rosario
2f70a57f18 Documentation: Update OCR and parsing indexing examples
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:27:01 -04:00
Roberto Rosario
5a626861ae Parsing: Add the 'content' attribute
Add the 'content' attribute to documents to allow access
to a document's parsed content for indexing and other purposes.

Fixes the document parsing indexing failing test.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:24:55 -04:00
Roberto Rosario
aaf9f7a8be OCR: Add 'ocr_content' attribute
Add the 'ocr_content' attribute to documents to allow access
to a document's OCR content for indexing and other purposes.

Fixes the OCR indexing failing test.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:20:31 -04:00
Roberto Rosario
0f5625a356 Tags: Update test indexing template for Jinja2
Jinja2 doesn't support the {% empty %} node, instead it uses
the {% else %} node for the same purpose.

Fixed the tag indexing failing test.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:16:18 -04:00
Roberto Rosario
cb408c768d PEP8: Code cleanups
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:26:56 -04:00
Roberto Rosario
03d51fe8e1 PEP8: Code cleanups
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:19:35 -04:00
Roberto Rosario
3394f97b25 Settings: Improve lazy values expression
Settings with lazy values are now more carefully checked
and converteed before serializing them.

Previously only the entire value was checked to see if it
was a promise. Now the value is checked to see if it is a
list or tuple and its members checked to see if they are
promises.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:17:59 -04:00
Roberto Rosario
7c6d466ab1 Settings: Expose new Django and Celery settings
Expose new Django settings via the UI: AUTH_PASSWORD_VALIDATORS,
DEBUG, DEFAULT_FROM_EMAIL, INTERNAL_IPS, LANGUAGES,
LANGUAGE_CODE, STATIC_URL, STATICFILES_STORAGE,
TIME_ZONE, WSGI_APPLICATION.

Expose a new Celery setting via the UI: CELERY_ALWAYS_EAGER.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:15:52 -04:00
Roberto Rosario
ba4858e77f Initial settings: Refactor setting bootstrapping
Refactor the initial environment settings and configuration file loading
fixing some issues loading Django settings.

Consolidate all database settings into a new single setting
called "DATABASES". This mirrors Django database setting
structure. This changes makes it possible to use configure
multiple databases and database routers from the environment
variables or configuration file.

Remove usage of django-environ. Only a small set of the
features provided by django-environ were being used.
Variable typecasting is now only YAML. YAML parsing
is implemented in code.

Previously the initial setting code added all settings
it found into the global symbol table. Now the settings
found are matched to a explicit list of allowed settings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:06:21 -04:00
Roberto Rosario
544edea54a Mailer: Code cleanup
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:51:08 -04:00
Roberto Rosario
2738e3facf Converter: Move literal to the literls.py module
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:50:31 -04:00
Roberto Rosario
51f15a3131 Settings: Update defaults formats
Update the default values of the settings which pass
arguments to backends to be valid Python values and not
YAML strings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:45:43 -04:00
Roberto Rosario
5c1eb59a1a Converter: Rename setting
Change the CONVERTER_GRAPHICS_BACKEND_CONFIG settings
to CONVERTER_GRAPHICS_BACKEND_ARGUMENTS for uniformity
and to convey it real purpose which is initializing a backend.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:39:48 -04:00
Roberto Rosario
f5f6470697 Settings: Make namespace a navigation link
Currently the link to navigate back to the settings
namespace is registered as an action when it is a link.
This commits changes it to a link in the facet menu.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:29:56 -04:00
Roberto Rosario
d5224d93a7 Settings: Remove support for quoted settings
Instead of passing strings as arguments to backends, all settings must
be formatted according to YAML specifications. This is to remove the
need to add separate YAML parsing to each backend argument in each
app that needs it. Argument passing to backends is not fully
uniform.

Users need to update their config files.
  Example:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS: '{location: /home/rosarior/development/mayan-edms/mayan/media/document_storage}'

  must be changed to:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS:
      location: /home/rosarior/development/mayan-edms/mayan/media/document_storage

  Example 2:

    CONVERTER_GRAPHICS_BACKEND_CONFIG: '        {            libreoffice_path: /usr/bin/libreoffice,            pdftoppm_dpi:
    300,            pdftoppm_format: jpeg,            pdftoppm_path: /usr/bin/pdftoppm,            pdfinfo_path:
    /usr/bin/pdfinfo,            pillow_format: JPEG        }    '

  must be changed to:

    CONVERTER_GRAPHICS_BACKEND_CONFIG:
      libreoffice_path: /usr/bin/libreoffice
      pdftoppm_dpi: 300
      pdftoppm_format: jpeg
      pdftoppm_path: /usr/bin/pdftoppm
      pdfinfo_path: /usr/bin/pdfinfo
      pillow_format: JPEG

  Example 3:

    OCR_BACKEND_ARGUMENTS: ''

  must be changed to:

    OCR_BACKEND_ARGUMENTS: {}

  Settings that need to be updated are:

  - COMMON_SHARED_STORAGE_ARGUMENTS
  - CONVERTER_GRAPHICS_BACKEND_CONFIG
  - DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS
  - DOCUMENTS_STORAGE_BACKEND_ARGUMENTS
  - OCR_BACKEND_ARGUMENTS
  - SIGNATURES_STORAGE_BACKEND_ARGUMENTS
  - SOURCES_STAGING_FILE_CACHE_STORAGE_BACKEND_ARGUMENTS

  The following error will appear in the console if a setting is not yet
  updated to this new format::

      TypeError: type object argument after ** must be a mapping, not str

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:27:57 -04:00
Roberto Rosario
2047fb7b17 Development: Don't add dev apps blindly
Try to import the development apps before adding them
as installed apps.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 16:43:34 -04:00
Roberto Rosario
4b727662ef Settings: Update MIDDLEWARE_CLASSES to MIDDLEWARE
Update mentions of MIDDLEWARE_CLASSES to MIDDLEWARE in the
development and testing settings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 02:15:50 -04:00
Roberto Rosario
6a7cd09bc1 Use Jinja2 as the template engine
Use Jinja2 to render the templates of the indexing,
workflows, smart links, user mailer and metadata apps.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 02:13:26 -04:00
Roberto Rosario
42d434f7bb Upload Wizard: Add double click support
Add support to select the document type using a Double Click action
on the form.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 01:34:44 -04:00
Roberto Rosario
e72b4e82a3 Forms: Add support for form hotkeys
Adds JavaScript support to monitor keypresses or mouse events
of forms with the classes .form-hotkey-enter or form-hotkey-double-click,
and trigger the click event of the button with the CSS class
.btn-hotkey-default.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 01:32:22 -04:00
Roberto Rosario
8e896a54f9 Middleware: Modernize middleware classes
Make the custom middleware provided by Mayan to use the
MiddlewareMixin provide by Django. This make the middleware
classes behave like classes or callables. This change ensures
compatibility with Django 2.x.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 00:59:39 -04:00
Roberto Rosario
2d5a646940 URLs: Remove development URLs from main URL file
Move the development URL definitions for Rosetta and Debug toolbar
to a separate URL file. Convert the single urls.py to a module to
allow multiple URL files to be used.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 00:32:17 -04:00
Roberto Rosario
b3c0b622b8 App: Remove admindocs
Remove admindocs which is not used in production.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-24 22:59:16 -04:00
Roberto Rosario
b04b205fb6 Add docstrings for almost all models
Also adds docstring to some managers and model methods.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-24 22:56:35 -04:00
Roberto Rosario
8c98679687 Dependencies: Remove django-suit from apps
Signed-off-by: Roberto Rosario <Roberto.Rosario.Gonzalez@gmail.com>
2018-11-22 04:59:10 -04:00
Roberto Rosario
6d39f3b716 Documents: Add missing import
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 22:02:33 -04:00
Roberto Rosario
21a6ed4756 Documents: Add document page icons
Add icons to the document page image and document page reset
views.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:58:10 -04:00
Roberto Rosario
3b247bfb5f Documents: Document task transformations
Add support to pass serialized transformation lists to the
document page generation task.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:51:57 -04:00
Roberto Rosario
823083a76c Converter: Add transformation serialization
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:51:32 -04:00
Roberto Rosario
4ee6add201 Documents: Document image API transformations
Add transformations support to the document image API.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:43:49 -04:00
Roberto Rosario
894a25ccce Appearance: Allow subclassing the TextArea widget
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:55:47 -04:00
Roberto Rosario
ecdc4a9a51 Appearance: Remove unused form_empty_label flag
This is superseded by the empty results template.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:48:32 -04:00