Main Content

Magnolia Community Forums: Development: Problem with imaging module on clustered environment


  • gzalys
    gzalys
    Full name: Gediminas Zalys
    Posts: 27
    Last post: Nov 23, 2016 7:04:41 AM
    Registered on: Aug 13, 2015
    Problem with imaging module on clustered environment
    #1 by gzalys on Jul 28, 2016 7:54:49 AM

    We have two public instances, that are clustered and are using NFS shared repository directory (/repositories/magnolia/repository/)
    The cluster ID is specified in magnolia.properties file for both public instances. We used samuelschmitt.net/2013/09/step-by-step-magnolia-cms-clustering/ guide to achieve clustering.
    Everything seems to be working fine apart from images.


    We keep on getting below error messages each time a call is made to retrieve an image.


    2016-07-28 12:49:01,704 ERROR info.magnolia.module.cache.filter.CacheFilter : A request started to cache but failed with an exception (ImagingException: Can't load image from /places/KFYIR5SWED5a/KFYIR5SWED5a_0_537x720.jpg/jcr:content). [url=http://uatloadbalancer-1471990206.ap-southeast-1.elb.amazonaws.com:8080/uatPublic/.imaging/hero_x3/dam/places/KFYIR5SWED5a/KFYIR5SWED5a_0_537x720.jpg]
    2016-07-28 12:49:01,704 ERROR info.magnolia.module.cache.filter.CacheFilter : A request started to cache but failed with an exception (ImagingException: Can't load image from /places/OPPIR5TY9POs/OPPIR5TY9POs_0_1440x1920.jpg/jcr:content). [url=http://uatloadbalancer-1471990206.ap-southeast-1.elb.amazonaws.com:8080/uatPublic/.imaging/hero_x3/dam/places/OPPIR5TY9POs/OPPIR5TY9POs_0_1440x1920.jpg]
    2016-07-28 12:49:01,704 ERROR info.magnolia.module.cache.filter.CacheFilter : A request started to cache but failed with an exception (ImagingException: Can't load image from /places/TRSYIR5S0KGXm/TRSYIR5S0KGXm_0_720x540.jpg/jcr:content). [url=http://uatloadbalancer-1471990206.ap-southeast-1.elb.amazonaws.com:8080/uatPublic/.imaging/hero_x3/dam/places/TRSYIR5S0KGXm/TRSYIR5S0KGXm_0_720x540.jpg]


    Whats interesting, is that we get this only on one of the instances (magnoliaPublic 2). Public 2 is NFS mounted onto public 1.
    Another point to add, we noticed this error message is thrown only once for one image. When you attempt to load this image again, it tends to work, however this behaviour is intermittent.

    Please also note that we are using load balancer to route the traffic to public 1 or public 2.

    Any advice will be appreciated.

  • runger
    runger
    Full name: Richard Unger
    Posts: 571
    Last post: Feb 1, 2017 9:54:59 AM
    Re: Problem with imaging module on clustered environment
    #2 by runger on Aug 1, 2016 1:15:54 PM

    Hi Gediminas,

    The imaging workspace is a pure cache for the generated image-variations. For this reason I would not cluster the imaging workspace. You will have more reliable and faster operation by having each public node have its own imaging workspace.

    Note also that there are other workspaces (e.g. mgnlSystem) which should not be clustered between the nodes.

    Regards from Vienna,

    Richard

  • gzalys
    gzalys
    Full name: Gediminas Zalys
    Posts: 27
    Last post: Nov 23, 2016 7:04:41 AM
    Registered on: Aug 13, 2015
    Re: Problem with imaging module on clustered environment
    #3 by gzalys on Aug 10, 2016 3:38:24 AM

    Thanks Richard!

    Would this mean i would have to put imaging module and others e.g. mgnlSystem to their own repository? and specify the location of that repository to sit in the private file system and separate DB location

    Best,
    Ged

  • gzalys
    gzalys
    Full name: Gediminas Zalys
    Posts: 27
    Last post: Nov 23, 2016 7:04:41 AM
    Registered on: Aug 13, 2015
    Re: Problem with imaging module on clustered environment
    #4 by gzalys on Aug 11, 2016 4:22:00 AM

    just to add on my previous message:

    From what i understand the below workspaces should sit in their own private space?

    <Map name="mgnlSystem" repositoryName="magnolia" workspaceName="mgnlSystem" />
    <Map name="mgnlVersion" repositoryName="magnolia" workspaceName="mgnlVersion" />
    <Map name="imaging" repositoryName="magnolia" workspaceName="imaging" />


    I use mysql for data storage, does it mean that these workspaces will need their own database each per instance ? or can i just use derby? would that be production safe?

  • runger
    runger
    Full name: Richard Unger
    Posts: 571
    Last post: Feb 1, 2017 9:54:59 AM
    Re: Problem with imaging module on clustered environment
    #5 by runger on Aug 17, 2016 12:19:49 PM

    Hi,

    Sorry for the delayed response, here's my take:

    - yes, the workspaces you mention should be local, no need to cluster them
    - the mgnlVersion workspace may be needed on each repository where you use versioning
    - each repository needs different storage backends, it’s a bit complicated. But basically you can use mysql if you want, or derby if you feel happy using it in production, but each repository needs its own database for this type of setup. If you already have a redundant database server system, you can just use that. Or you can just use a local mysql instance on each node.

    Regards from Vienna,

    Richard

  • gzalys
    gzalys
    Full name: Gediminas Zalys
    Posts: 27
    Last post: Nov 23, 2016 7:04:41 AM
    Registered on: Aug 13, 2015
    Re: Problem with imaging module on clustered environment
    #6 by gzalys on Aug 20, 2016 8:17:06 AM

    Many thanks for your support Richard!

    I feel that derby is not a wise choice for production systems, so ill leave this option out.

    I am slightly concerned in regards to managing this setup.
    Lets say i have 10 nodes running magnolia public instances. This means i will need 10 jackrabbit files pointing to different mysql databases.
    And also what about backups? backing up that many databases will surely eventually create inconstancies.

    feels to me that data will end up all over the place.

    maybe there is a way to say use one jackrabbit file, that points to one DB, however create tables with different prefixes, i.e node1_ mgnlVersion etc?

    Thanks again for your help,
    Ged

  • runger
    runger
    Full name: Richard Unger
    Posts: 571
    Last post: Feb 1, 2017 9:54:59 AM
    Re: Problem with imaging module on clustered environment
    #7 by runger on Aug 29, 2016 11:34:23 AM

    Hi Ged,

    The questions you're asking are in the direction of operational management of your mangolia setup. They are good questions, and I don't know if there is a perfect answer - it depends on your operations team, server infrastructure, etc...

    For the Jackrabbit files, I think a good approach is to use a "generic" file that has variables in it. This file can be part of the deployed webapp. The variable values can be defined differently for each server using a magnolia.properties config file. I find it best to keep this .properties file outside the webapp, e.g. it isn't part of the deployment, it is part of the server setup. tomcat/conf can be a natural place to store it. You can define where to find it to magnolia using a -D parameter.

    In this way all your nodes use the same jackrabbit configuration, but with different parameters like database host or schema name.

    Regarding mysql setup, you have the full range of options: separate servers, separate databases, or different prefixes within the same database. Personally I don't see the advantage of putting everything in the same database, but maybe there is a good reason to do so.

    Backing up your magnolia instances is a separate problem again. Really, a Jackrabbit Repository has to be stopped to guarantee a consistent backup at the database/datastore level. This means that you have to shut down the magnolia instance using it if you want to create a backup at the database level.
    Some ideas for dealing with this:
    1) some people back up on the JCR level rather than the database level, by using the JCR export functions. This then does not require a magnolia shutdown.
    2) you can consider not backing up your public nodes, or at least not regularly. This is especially true if you have multiple public servers. If you have a fast way to restore a "naked" public node, the data can then be restored by publishing from the author instance (using the synchronization module, or manually).
    3) and for your author instance, it is usually ok to shut it down at night in order to create a consistent backup...

    If you are running a lot of magnolia instances, it really pays to take the time to think about these issues, and the management of the servers. By using virtualization or containerization, separating the per-node configuration from the webapp, and automating deployments you can really save a lot of time and effort in a large installation.

    Regards from Vienna,

    Richard

You don't have the permission to post on this thread

Sign in

To login on this forum, you can use your Magnolia Forge, Support or Partner account, or, below, your Google, Yahoo! or OpenID account. If you have trouble logging in, or any other sort of issue, please let us know in the Meta forum, on the user-list, or simply by email at forum-admin at magnolia-cms dot com.

* Required

... or sign in with:

  • icon http://{your-openid-url}
  • icon
  • icon https://me.yahoo.com/