Commit Graph
57 Commits
Author SHA1 Message Date
Roberto Rosario 8e66eefe7c Move file and storage code to the storage app
The setting COMMON_TEMPORARY_DIRECTORY is now
STORAGE_TEMPORARY_DIRECTORY.

Signed-off-by: Roberto Rosario <Roberto.Rosario@mayan-edms.com>
2019-01-31 22:30:51 -04:00
Roberto Rosario da6d7cbc0c Converter: Sort ConverterBase methods
Signed-off-by: Roberto Rosario <Roberto.Rosario@mayan-edms.com>
2018-12-08 16:34:34 -04:00
Roberto Rosario 27c04ed9be Converter: Remove base64 image support
The get_page method had support to return the image in
base64 format. This feature is no longer used by any
other app.

Signed-off-by: Roberto Rosario <Roberto.Rosario@mayan-edms.com>
2018-12-05 14:47:12 -04:00
Roberto Rosario 8e69178e07 Project: Switch to full app paths
Instead of inserting the path of the apps into the Python app,
the apps are now referenced by their full import path.

This app name claves with external or native Python libraries.
Example: Mayan statistics app vs. Python new statistics library.

Every app reference is now prepended with 'mayan.apps'.

Existing config.yml files need to be updated manually.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-12-05 02:04:20 -04:00
Roberto Rosario 2738e3facf Converter: Move literal to the literls.py module
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:50:31 -04:00
Roberto Rosario 5c1eb59a1a Converter: Rename setting
Change the CONVERTER_GRAPHICS_BACKEND_CONFIG settings
to CONVERTER_GRAPHICS_BACKEND_ARGUMENTS for uniformity
and to convey it real purpose which is initializing a backend.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:39:48 -04:00
Roberto Rosario d5224d93a7 Settings: Remove support for quoted settings
Instead of passing strings as arguments to backends, all settings must
be formatted according to YAML specifications. This is to remove the
need to add separate YAML parsing to each backend argument in each
app that needs it. Argument passing to backends is not fully
uniform.

Users need to update their config files.
  Example:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS: '{location: /home/rosarior/development/mayan-edms/mayan/media/document_storage}'

  must be changed to:

    DOCUMENTS_STORAGE_BACKEND_ARGUMENTS:
      location: /home/rosarior/development/mayan-edms/mayan/media/document_storage

  Example 2:

    CONVERTER_GRAPHICS_BACKEND_CONFIG: '        {            libreoffice_path: /usr/bin/libreoffice,            pdftoppm_dpi:
    300,            pdftoppm_format: jpeg,            pdftoppm_path: /usr/bin/pdftoppm,            pdfinfo_path:
    /usr/bin/pdfinfo,            pillow_format: JPEG        }    '

  must be changed to:

    CONVERTER_GRAPHICS_BACKEND_CONFIG:
      libreoffice_path: /usr/bin/libreoffice
      pdftoppm_dpi: 300
      pdftoppm_format: jpeg
      pdftoppm_path: /usr/bin/pdftoppm
      pdfinfo_path: /usr/bin/pdfinfo
      pillow_format: JPEG

  Example 3:

    OCR_BACKEND_ARGUMENTS: ''

  must be changed to:

    OCR_BACKEND_ARGUMENTS: {}

  Settings that need to be updated are:

  - COMMON_SHARED_STORAGE_ARGUMENTS
  - CONVERTER_GRAPHICS_BACKEND_CONFIG
  - DOCUMENTS_CACHE_STORAGE_BACKEND_ARGUMENTS
  - DOCUMENTS_STORAGE_BACKEND_ARGUMENTS
  - OCR_BACKEND_ARGUMENTS
  - SIGNATURES_STORAGE_BACKEND_ARGUMENTS
  - SOURCES_STAGING_FILE_CACHE_STORAGE_BACKEND_ARGUMENTS

  The following error will appear in the console if a setting is not yet
  updated to this new format::

      TypeError: type object argument after ** must be a mapping, not str

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-11-26 17:27:57 -04:00
Roberto Rosario c6e68515ed Handle office files in explicit binary mode for Python 3.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-10-06 05:07:05 -04:00
Roberto Rosario d8ade58472 Add explicit kwargs name.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2018-09-03 12:17:01 -04:00
Michael Price ee58303fbb Move transformations to the their own module.
Signed-off-by: Michael Price <loneviking72@gmail.com>
2018-04-01 20:18:54 -04:00
Roberto Rosario 9dba40a1d8 Use BytesIO instead of StringIO. Use text_type instead of unicode.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-08-29 02:12:18 -04:00
Roberto Rosario b6ffeaeb32 Allow changing the output format, DPI of the pdftoppm command, and
the output format of the converter via the CONVERTER_GRAPHICS_BACKEND_CONFIG
setting sub options: pdftoppm_dpi: 300, pdftoppm_format: jpeg, pillow_format: jpeg
GitHub issues #256 #257 GitLab issue #416.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-08-03 01:10:39 -04:00
Roberto Rosario 32c53343f1 Fix converter bug introduced in previous commit merge.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-07-05 01:03:14 -04:00
Roberto Rosario b0912947fc Give every libreoffice instance its own separate $HOME directory.
Additionally give every libreoffice its own UserInstallation
file in the $HOME directory. Works around Libre Office issue:
https://bugs.documentfoundation.org/show_bug.cgi?id=37531
Solves or affects GitLab issues #393 #258 #198 #175

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-07-04 23:58:04 -04:00
Roberto Rosario 7343223f59 PDF compatibility improvements. Support simple encrypted PDF.
Improve PDF page count detection. Improve PDF orientation
detection. Merge CONVERTER_LIBREOFFICE_PATH and
CONVERTER_PDFTOPPM_PATH config settings into
CONVERTER_GRAPHICS_BACKEND_CONFIG.
GitLab issue #384 #376 #377.

Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-07-04 04:02:29 -04:00
Roberto Rosario 6bad5e02d1 Add lineart transformation.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-07-03 14:05:13 -04:00
Roberto Rosario 4150fea2ad Add PDF orientation detection. Closes GitLab issue #387.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-06-16 22:30:07 -04:00
Roberto Rosario 7a0c2224bb Add model docstrings.
Signed-off-by: Roberto Rosario <roberto.rosario.gonzalez@gmail.com>
2017-04-25 14:21:30 -04:00
Roberto Rosario 92d2f9cbdd Add mirror, flip, gaussian blur and unsharpen masking transformations. 2017-01-18 03:01:27 -04:00
Roberto Rosario 9cb5758245 Add 3 new "prepared" transformations: Rotate 90, 180 and 270 degrees.
Sort transformation list.
2017-01-18 02:35:14 -04:00
Roberto Rosario ba11e63364 Improve transformation hashing algorithm. 2016-11-16 19:11:55 -04:00
Roberto Rosario aa2e8f3a11 Merge remote-tracking branch 'origin/master' into merge_test 2016-11-09 01:27:34 -04:00
Roberto Rosario 98327ae877 Return the same image when rotation degrees are 0. 2016-11-08 17:47:51 -04:00
Roberto Rosario 4222035cc1 Backport zoom transformation performance improvement, GitLab issue #334. Thanks to @startmat for the patch. 2016-11-08 02:03:10 -04:00
Roberto Rosario f98063835a Backport resize transformation math operation fix, GitLab issue #319. Thanks to @startmat for the find. 2016-11-08 01:51:27 -04:00
Roberto Rosario 19d20ff9d1 Prevent unnecessary image.resize in TransformationZoom. Thanks to @startmat for the suggestion. GitLab issue #334. 2016-11-05 02:56:43 -04:00
Roberto Rosario 621c988809 Simplify filter_by_access boilerplate code 2016-11-02 03:48:16 -04:00
Roberto Rosario c0194c63dc Refactor document page image generation and transformation classes
to cache all transformed versions of a document page.
2016-11-02 02:34:57 -04:00
Roberto Rosario de6564c264 Fix height calculation and remove disabled "fit" code. Thanks to @startmat for finding this one. Closes GitLab issue #319. 2016-10-31 15:08:43 -04:00
Roberto Rosario f3f5cff36e Add missing temporary cleanup for the office documents section. 2016-06-28 01:53:53 -04:00
Roberto Rosario 945158bd60 Normalize handling of temporary file and directory creation. 2016-06-23 20:54:10 -04:00
Roberto Rosario 28f3b7551d Consolidate document page image generation as binary or base64 to a single place at the converter class level. 2016-03-21 03:44:45 -04:00
Roberto Rosario 923ecc85d8 PEP8 Cleanups. 2015-10-28 02:22:10 -04:00
Roberto Rosario 86ff835839 PEP8 Cleanups. 2015-10-07 22:24:44 -04:00
Roberto Rosario 3d3abb6ad6 Use SH to call LibreOffice binary. Force LibreOffice input filter when converting text files. 2015-09-26 03:00:11 -04:00
Roberto Rosario dbd676646b Don't convert images blindly to RGB. Only convert to RGB or RGBA when saving to JPEG and mode is 'P'. documents.tests.test_models.MultiPageTiffTestCase passes again. 2015-09-16 15:56:10 -04:00
Roberto Rosario d6a3111d69 Make sure all images are converted to RGB before processing. 2015-09-15 02:01:56 -04:00
Roberto Rosario 59696ba68d Add missing exception variable. 2015-08-23 23:10:25 -04:00
Roberto Rosario d1bb06f350 Add logging to mark instances where getting the page count of a file may fails because it is not an office format file. 2015-08-23 22:47:33 -04:00
Roberto Rosario 5bb8f779b3 Add crop document page transformation. 2015-08-12 17:23:36 -04:00
Roberto Rosario 1e746c700a PEP8 cleanups. 2015-07-27 23:53:14 -04:00
Roberto Rosario 4527563d89 PEP8 cleanups, specially E501 line too long. 2015-07-22 18:21:37 -04:00
Roberto Rosario c275988c90 PEP8 cleanups, E501: 79 character long lines. 2015-07-19 03:27:41 -04:00
Roberto Rosario fa4cb686fb Auto add arguments to transformation labels. 2015-07-04 04:16:18 -04:00
Roberto Rosario 862de43219 Update tranformation help string. 2015-07-04 04:01:20 -04:00
Roberto Rosario 3d7e7ae4a2 Add support for getting an office document page count. 2015-07-03 03:19:42 -04:00
Roberto Rosario 48df3dcafa PEP8 cleanups 2015-06-24 17:11:24 -04:00
Roberto Rosario b18888b3f7 Convert and cache office documents at the document version level for faster page image retrieval 2015-06-24 01:04:35 -04:00
Roberto Rosario e4623fadcd PEP8 cleanups 2015-06-23 02:23:23 -04:00
Roberto Rosario 78198f3398 Smart settings refactor 2015-06-22 21:04:06 -04:00