Commit Graph

6836 Commits

Author SHA1 Message Date
Roberto Rosario
2ca38c20b0 Tests: Fix failing tests
Fix failing tests in the OCR and parsing apps.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 17:17:07 -04:00
Roberto Rosario
67e79d0e19 OCR, Parsing: Revert iterator stop
Revert how the OCR and document parsing generators end
their iteration. Originally they issue an empty return,
then a blank yield was added. This commit reverts the
blank yield and restores the original 'return' behavior.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 17:15:38 -04:00
Roberto Rosario
141d79afa6 Documentation: Add initial 3.2 release notes file
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:31:18 -04:00
Roberto Rosario
e9411514c7 PEP8: Code cleanup
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:28:55 -04:00
Roberto Rosario
2f70a57f18 Documentation: Update OCR and parsing indexing examples
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:27:01 -04:00
Roberto Rosario
5a626861ae Parsing: Add the 'content' attribute
Add the 'content' attribute to documents to allow access
to a document's parsed content for indexing and other purposes.

Fixes the document parsing indexing failing test.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:24:55 -04:00
Roberto Rosario
aaf9f7a8be OCR: Add 'ocr_content' attribute
Add the 'ocr_content' attribute to documents to allow access
to a document's OCR content for indexing and other purposes.

Fixes the OCR indexing failing test.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:20:31 -04:00
Roberto Rosario
0f5625a356 Tags: Update test indexing template for Jinja2
Jinja2 doesn't support the {% empty %} node, instead it uses
the {% else %} node for the same purpose.

Fixed the tag indexing failing test.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 05:16:18 -04:00
Roberto Rosario
cb408c768d PEP8: Code cleanups
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:26:56 -04:00
Roberto Rosario
03d51fe8e1 PEP8: Code cleanups
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:19:35 -04:00
Roberto Rosario
3394f97b25 Settings: Improve lazy values expression
Settings with lazy values are now more carefully checked
and converteed before serializing them.

Previously only the entire value was checked to see if it
was a promise. Now the value is checked to see if it is a
list or tuple and its members checked to see if they are
promises.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:17:59 -04:00
Roberto Rosario
7c6d466ab1 Settings: Expose new Django and Celery settings
Expose new Django settings via the UI: AUTH_PASSWORD_VALIDATORS,
DEBUG, DEFAULT_FROM_EMAIL, INTERNAL_IPS, LANGUAGES,
LANGUAGE_CODE, STATIC_URL, STATICFILES_STORAGE,
TIME_ZONE, WSGI_APPLICATION.

Expose a new Celery setting via the UI: CELERY_ALWAYS_EAGER.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:15:52 -04:00
Roberto Rosario
ba4858e77f Initial settings: Refactor setting bootstrapping
Refactor the initial environment settings and configuration file loading
fixing some issues loading Django settings.

Consolidate all database settings into a new single setting
called "DATABASES". This mirrors Django database setting
structure. This changes makes it possible to use configure
multiple databases and database routers from the environment
variables or configuration file.

Remove usage of django-environ. Only a small set of the
features provided by django-environ were being used.
Variable typecasting is now only YAML. YAML parsing
is implemented in code.

Previously the initial setting code added all settings
it found into the global symbol table. Now the settings
found are matched to a explicit list of allowed settings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:06:21 -04:00
Roberto Rosario
544edea54a Mailer: Code cleanup
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:51:08 -04:00
Roberto Rosario
2738e3facf Converter: Move literal to the literls.py module
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:50:31 -04:00
Roberto Rosario
51f15a3131 Settings: Update defaults formats
Update the default values of the settings which pass
arguments to backends to be valid Python values and not
YAML strings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:45:43 -04:00
Roberto Rosario
5c1eb59a1a Converter: Rename setting
Change the CONVERTER_GRAPHICS_BACKEND_CONFIG settings
to CONVERTER_GRAPHICS_BACKEND_ARGUMENTS for uniformity
and to convey it real purpose which is initializing a backend.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:39:48 -04:00
Roberto Rosario
f5f6470697 Settings: Make namespace a navigation link
Currently the link to navigate back to the settings
namespace is registered as an action when it is a link.
This commits changes it to a link in the facet menu.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:29:56 -04:00
Roberto Rosario
d5224d93a7 Settings: Remove support for quoted settings
Instead of passing strings as arguments to backends, all settings must
be formatted according to YAML specifications. This is to remove the
need to add separate YAML parsing to each backend argument in each
app that needs it. Argument passing to backends is not fully
uniform.

Users need to update their config files.
  Example:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS: '{location: /home/rosarior/development/mayan-edms/mayan/media/document_storage}'

  must be changed to:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS:
      location: /home/rosarior/development/mayan-edms/mayan/media/document_storage

  Example 2:

    CONVERTER_GRAPHICS_BACKEND_CONFIG: '        {            libreoffice_path: /usr/bin/libreoffice,            pdftoppm_dpi:
    300,            pdftoppm_format: jpeg,            pdftoppm_path: /usr/bin/pdftoppm,            pdfinfo_path:
    /usr/bin/pdfinfo,            pillow_format: JPEG        }    '

  must be changed to:

    CONVERTER_GRAPHICS_BACKEND_CONFIG:
      libreoffice_path: /usr/bin/libreoffice
      pdftoppm_dpi: 300
      pdftoppm_format: jpeg
      pdftoppm_path: /usr/bin/pdftoppm
      pdfinfo_path: /usr/bin/pdfinfo
      pillow_format: JPEG

  Example 3:

    OCR_BACKEND_ARGUMENTS: ''

  must be changed to:

    OCR_BACKEND_ARGUMENTS: {}

  Settings that need to be updated are:

  - COMMON_SHARED_STORAGE_ARGUMENTS
  - CONVERTER_GRAPHICS_BACKEND_CONFIG
  - DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS
  - DOCUMENTS_STORAGE_BACKEND_ARGUMENTS
  - OCR_BACKEND_ARGUMENTS
  - SIGNATURES_STORAGE_BACKEND_ARGUMENTS
  - SOURCES_STAGING_FILE_CACHE_STORAGE_BACKEND_ARGUMENTS

  The following error will appear in the console if a setting is not yet
  updated to this new format::

      TypeError: type object argument after ** must be a mapping, not str

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:27:57 -04:00
Roberto Rosario
2047fb7b17 Development: Don't add dev apps blindly
Try to import the development apps before adding them
as installed apps.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 16:43:34 -04:00
Roberto Rosario
4b727662ef Settings: Update MIDDLEWARE_CLASSES to MIDDLEWARE
Update mentions of MIDDLEWARE_CLASSES to MIDDLEWARE in the
development and testing settings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 02:15:50 -04:00
Roberto Rosario
6a7cd09bc1 Use Jinja2 as the template engine
Use Jinja2 to render the templates of the indexing,
workflows, smart links, user mailer and metadata apps.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 02:13:26 -04:00
Roberto Rosario
42d434f7bb Upload Wizard: Add double click support
Add support to select the document type using a Double Click action
on the form.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 01:34:44 -04:00
Roberto Rosario
e72b4e82a3 Forms: Add support for form hotkeys
Adds JavaScript support to monitor keypresses or mouse events
of forms with the classes .form-hotkey-enter or form-hotkey-double-click,
and trigger the click event of the button with the CSS class
.btn-hotkey-default.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 01:32:22 -04:00
Roberto Rosario
8e896a54f9 Middleware: Modernize middleware classes
Make the custom middleware provided by Mayan to use the
MiddlewareMixin provide by Django. This make the middleware
classes behave like classes or callables. This change ensures
compatibility with Django 2.x.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 00:59:39 -04:00
Roberto Rosario
2d5a646940 URLs: Remove development URLs from main URL file
Move the development URL definitions for Rosetta and Debug toolbar
to a separate URL file. Convert the single urls.py to a module to
allow multiple URL files to be used.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-25 00:32:17 -04:00
Roberto Rosario
b3c0b622b8 App: Remove admindocs
Remove admindocs which is not used in production.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-24 22:59:16 -04:00
Roberto Rosario
b04b205fb6 Add docstrings for almost all models
Also adds docstring to some managers and model methods.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-24 22:56:35 -04:00
Roberto Rosario
8c98679687 Dependencies: Remove django-suit from apps
Signed-off-by: Roberto Rosario <Roberto.Rosario.Gonzalez@gmail.com>
2018-11-22 04:59:10 -04:00
Roberto Rosario
6d39f3b716 Documents: Add missing import
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 22:02:33 -04:00
Roberto Rosario
21a6ed4756 Documents: Add document page icons
Add icons to the document page image and document page reset
views.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:58:10 -04:00
Roberto Rosario
3b247bfb5f Documents: Document task transformations
Add support to pass serialized transformation lists to the
document page generation task.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:51:57 -04:00
Roberto Rosario
823083a76c Converter: Add transformation serialization
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:51:32 -04:00
Roberto Rosario
4ee6add201 Documents: Document image API transformations
Add transformations support to the document image API.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:43:49 -04:00
Roberto Rosario
894a25ccce Appearance: Allow subclassing the TextArea widget
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:55:47 -04:00
Roberto Rosario
ecdc4a9a51 Appearance: Remove unused form_empty_label flag
This is superseded by the empty results template.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:48:32 -04:00
Roberto Rosario
2ae56d2cf4 Dependencies: Update Python dependencies version.
Remove django suit as a dependency.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:41:02 -04:00
Roberto Rosario
810558659d Documents: Add invalid document server template
Invalid document template is now served or included from
a specific template file. Documents with invalid
API image URLs now return None instead of the template code
specific '#'.  The new template is called invalid_document.html.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:23:43 -04:00
Roberto Rosario
957cf64fe5 Pagination: Add custom pure pagination subclasses
Add Paginator and Page subclasses that supports custom page
querystring keys.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:04:42 -04:00
Roberto Rosario
e8c70cbd08 Pagination: Fix unintented pagination AJAX reload
Add CSS class to disable reload when clicking on pagination active page
link.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 17:59:28 -04:00
Roberto Rosario
8cbae9021b Release: Update setup build string
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-01 03:55:42 -04:00
Roberto Rosario
8a971400b1 Release: Bump version to 3.1.9
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-01 03:55:06 -04:00
Roberto Rosario
fb3678bb6f Sources: Convert furl querystring to text
Convert the furl instance to text to allow serializing it into
JSON to be passed as arguments to the background task.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-01 03:50:34 -04:00
Roberto Rosario
7c5477b439 Release: Update build string
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 19:10:08 -04:00
Roberto Rosario
cc7362b269 Release: Bump version to 3.1.8
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 19:09:05 -04:00
Roberto Rosario
dcb55f4aff Utilities: Remove remarked statement
Remove print statement fromt he increase_version module.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 19:05:47 -04:00
Roberto Rosario
ba6fd842d2 Documentation: Add version 3.1.8 release notes
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 19:05:03 -04:00
Roberto Rosario
cf1726505b Documenation: Fix spelling errors
Fix spelling errors in documentation and other program texts.
Update the spelling exceptions list.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 18:41:20 -04:00
Roberto Rosario
8b41a9bfc9 Documentation: Expand the Source control section
Add new subsection explaining commit messages structure to be
used in the project.

Updates and expands explanations regarding the Git branch
structure used in the project.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 17:24:43 -04:00
Roberto Rosario
e109068b29 Indexing: Add document base property reindex
Add support for reindexing documents when their base properties like
the label and description are edited.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 17:08:56 -04:00