Commit Graph

942 Commits

Author SHA1 Message Date
Roberto Rosario
c466c44dfb Search: Fix failing tests
The SearchModel class no longer returns an elapsed time
value. Update the tests and views that expect this removed
variable.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-05 03:19:27 -04:00
Roberto Rosario
55cd928069 Documents: Add default filtering of stubs
Add filter(is_stub) to the default Document model manager.

Now only the Passthrough manager can access document stubs.

Remove the explicit filtering of stubs from code that obtains
the queryset from the default document manager.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-05 03:05:39 -04:00
Roberto Rosario
2d4a710999 Document: Rename DocumentPageResult model
Rename the DocumentPageResult to DocumentPageSearchResult for
clarity of purpose.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-05 02:48:34 -04:00
Roberto Rosario
8e69178e07 Project: Switch to full app paths
Instead of inserting the path of the apps into the Python app,
the apps are now referenced by their full import path.

This app name claves with external or native Python libraries.
Example: Mayan statistics app vs. Python new statistics library.

Every app reference is now prepended with 'mayan.apps'.

Existing config.yml files need to be updated manually.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-05 02:04:20 -04:00
Roberto Rosario
146ebb7032 Documents: Update cache size on setting update
Add a setting callback to update the size of the document
cache when the DOCUMENTS_CACHE_MAXIMUM_SIZE setting is
updated.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-04 15:48:44 -04:00
Roberto Rosario
f4e0e06c66 Documents: Remove old image caching model
With the creation of the new general use file Cache system
the old DocumentPageCachedImage model and manager are no longer
needed. This commit removed the model and the manager, and add
a migration to remove any data in the model before removing it
from the database.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-04 15:21:57 -04:00
Roberto Rosario
1d1b4f5f5f Documents: Organize methods per best practices
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-04 00:00:42 -04:00
Roberto Rosario
89d3fb9922 Cache: Beta stage
Add retry to race condition in intermediate image generation.

Remove DB index from cache file size field.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-03 04:26:00 -04:00
Roberto Rosario
77fc9b5831 Caching: MVP stage
Working implementation with three levels of organization per cache.
Cache -> Partition -> File.

Current non blocking lock for the intermediate file generation needs
to be replaced to a blocking lock.

Needs tests.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-02 05:00:35 -04:00
Roberto Rosario
a414b8df92 Caching: Initial experitmental cache model
signed-off-by: Roberto Rosario <rosarior@t60.lan>
2018-11-30 19:48:26 -04:00
Roberto Rosario
abfc8b0c09 Navigation: Add list facet menu
Add the new list facet navigation menu. Used to provide facets
to the list subtemplate. The list facet allows separating the object
action links from the object navigation links. The list facet
links are displayed before the list object links on the list
subtemplate. In the object views, the list facet menu behaves
exactly the same as the form facet menu.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-29 04:17:53 -04:00
Roberto Rosario
52bbf62e26 Documents: PEP8 and code style cleanups
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-29 02:11:26 -04:00
Roberto Rosario
cb408c768d PEP8: Code cleanups
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-27 04:26:56 -04:00
Roberto Rosario
51f15a3131 Settings: Update defaults formats
Update the default values of the settings which pass
arguments to backends to be valid Python values and not
YAML strings.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:45:43 -04:00
Roberto Rosario
d5224d93a7 Settings: Remove support for quoted settings
Instead of passing strings as arguments to backends, all settings must
be formatted according to YAML specifications. This is to remove the
need to add separate YAML parsing to each backend argument in each
app that needs it. Argument passing to backends is not fully
uniform.

Users need to update their config files.
  Example:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS: '{location: /home/rosarior/development/mayan-edms/mayan/media/document_storage}'

  must be changed to:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS:
      location: /home/rosarior/development/mayan-edms/mayan/media/document_storage

  Example 2:

    CONVERTER_GRAPHICS_BACKEND_CONFIG: '        {            libreoffice_path: /usr/bin/libreoffice,            pdftoppm_dpi:
    300,            pdftoppm_format: jpeg,            pdftoppm_path: /usr/bin/pdftoppm,            pdfinfo_path:
    /usr/bin/pdfinfo,            pillow_format: JPEG        }    '

  must be changed to:

    CONVERTER_GRAPHICS_BACKEND_CONFIG:
      libreoffice_path: /usr/bin/libreoffice
      pdftoppm_dpi: 300
      pdftoppm_format: jpeg
      pdftoppm_path: /usr/bin/pdftoppm
      pdfinfo_path: /usr/bin/pdfinfo
      pillow_format: JPEG

  Example 3:

    OCR_BACKEND_ARGUMENTS: ''

  must be changed to:

    OCR_BACKEND_ARGUMENTS: {}

  Settings that need to be updated are:

  - COMMON_SHARED_STORAGE_ARGUMENTS
  - CONVERTER_GRAPHICS_BACKEND_CONFIG
  - DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS
  - DOCUMENTS_STORAGE_BACKEND_ARGUMENTS
  - OCR_BACKEND_ARGUMENTS
  - SIGNATURES_STORAGE_BACKEND_ARGUMENTS
  - SOURCES_STAGING_FILE_CACHE_STORAGE_BACKEND_ARGUMENTS

  The following error will appear in the console if a setting is not yet
  updated to this new format::

      TypeError: type object argument after ** must be a mapping, not str

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:27:57 -04:00
Roberto Rosario
6d39f3b716 Documents: Add missing import
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 22:02:33 -04:00
Roberto Rosario
21a6ed4756 Documents: Add document page icons
Add icons to the document page image and document page reset
views.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:58:10 -04:00
Roberto Rosario
3b247bfb5f Documents: Document task transformations
Add support to pass serialized transformation lists to the
document page generation task.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:51:57 -04:00
Roberto Rosario
4ee6add201 Documents: Document image API transformations
Add transformations support to the document image API.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 21:43:49 -04:00
Roberto Rosario
810558659d Documents: Add invalid document server template
Invalid document template is now served or included from
a specific template file. Documents with invalid
API image URLs now return None instead of the template code
specific '#'.  The new template is called invalid_document.html.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-16 18:23:43 -04:00
Roberto Rosario
e109068b29 Indexing: Add document base property reindex
Add support for reindexing documents when their base properties like
the label and description are edited.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 17:08:56 -04:00
Roberto Rosario
9fc7c4fc09 Add explicit argument name of 'mode' to the open statement.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-31 16:53:48 -04:00
Roberto Rosario
5a8455bfc2 Update translation files.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-29 13:24:07 -04:00
Roberto Rosario
a99b044555 Code style improvement. Test code consolidation. PEP8 cleanups.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-06 02:13:36 -04:00
Roberto Rosario
e9db07bfd3 Encapsulate in_trash field into is_in_trash symetric Document property
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-03 15:14:11 -04:00
Roberto Rosario
bbfe35b8fd Disable the fancybox caption link if the document is in the trash.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-02 02:53:55 -04:00
Roberto Rosario
f6e39380af Remove use of hard coded font icon for document page rendering busy indicator.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-02 02:40:13 -04:00
Roberto Rosario
26ac7de70b Synchronize and compile translations
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-26 22:50:48 -04:00
Roberto Rosario
3c2557fb47 Update translation source files.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-26 22:29:54 -04:00
Roberto Rosario
6159bdca45 Fix message typos.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-26 22:23:35 -04:00
Roberto Rosario
758a14e358 Intercept document list view exception and display them as an error message.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-21 02:10:20 -04:00
Roberto Rosario
7c4de79c8f Calculate the path of the test document before it is uploaded.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-21 00:47:35 -04:00
Roberto Rosario
a96e7574b2 Add support for natural keys to the DocumentPageImageCache model.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-21 00:47:11 -04:00
Roberto Rosario
eda8d18146 Database access in data migrations defaults to the 'default' database. Force it to the user selected database instead.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-19 20:53:04 -04:00
Roberto Rosario
a986b58338 Prepare release files.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 18:52:26 -04:00
Roberto Rosario
ecdc122b15 Code cleanups.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 18:28:34 -04:00
Roberto Rosario
58f7b1b555 Add file_size and datetime fields to the DocumentPageCachedImage model.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 03:39:28 -04:00
Roberto Rosario
206dbb9b5b The size of the document type label field has been increased from 32 to 96 characters.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 03:14:27 -04:00
Roberto Rosario
ac07d4a63f Add more icons to links.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 03:09:04 -04:00
Roberto Rosario
4d53c9aec7 Code clean up.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 02:54:47 -04:00
Roberto Rosario
a372fc5a07 Improve model help texts. Add respective migrations.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 02:46:16 -04:00
Roberto Rosario
03c54395cc Refactor the ModelAttribute class into two separate classes: ModelAttribute for executable model attributes and ModelField for actual ORM fields. Expose more document fields for use in smart links.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-17 02:43:04 -04:00
Roberto Rosario
55930689bb Update language files.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-12 03:45:51 -04:00
Roberto Rosario
962f395831 Add new dashboard item to display the total page count.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-12 01:56:37 -04:00
Roberto Rosario
4eab4670be Unify tests that use upload documents. Add explicit mode when opening a file.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-06 03:38:17 -04:00
Roberto Rosario
64ca5a623f PEP8 cleanups.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-05 04:56:39 -04:00
Roberto Rosario
435fa774f0 Add support for preserving the extension of document files when using the quick label feature. Added to the document properties edit view and the document upload view. Closes GitLab issue #360.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-05 04:06:13 -04:00
Roberto Rosario
aac0852128 Clean up document download view class.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-05 01:03:40 -04:00
Roberto Rosario
12143ffda7 Fix document type quick label link when there are no results.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-05 01:03:04 -04:00
Roberto Rosario
e5bb00d514 Finish refactor of the new compressed file class support. Closes GitLab issue #7.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-05 00:40:10 -04:00