For some context, I have a supervisor handling equipment operations for lead/lag, staging, mode calculation, etc. This information is then fed down to 2 Jaces via Niagara connections where the Jaces then execute the rest of the individual equipment logic. Due to some hardware restrictions and the redundant needs of the system in general, I’m left with multiple potential points of failure to account for within programming for situations where network Niagara connections are lost for whatever reason.
After some testing, I’ve managed to determine fallback by monitoring the Niagara connections using status demux’s. If the supervisor connection fails, it toggles numeric/boolean switches values coming from the supervisor, bound to the highest priority on associated values within the Jace, to null. This then allows the identical lead lag, staging, etc logic as the supervisor, hosted on the Jace to execute on the same points, just bound to a lower priority. It works in my test environment but it is a bit clunky and I’m not really happy with how transitioning back to normal operation is handled once connections are restored which basically requires fully shutting down the equipment and starting up from scratch. I’m curious if anyone would be willing to share ideas/examples of better or more efficient ways of solving this.
My name is Charles. I’m one of the moderators here for DDC-Talk!
The issue with using the StatusDemux is that your controller may still ping but can be locked up due to watchdog timeouts (especially if they have chosen logging instead of a reboot or terminate).
Here’s is the JACE redundancy document, which admittedly I don’t know much about since I haven’t had to do anything with redundant systems in quite some time.
Please feel free to read about it here.
Instead of a status demux, you could create a variable that spits out a random value in intervals and if another station doesn’t see a value change in that object, you could consider that JACE to be non-communicating.
Welcome to the community! Feel free to join our discord too!
Charles - I appreciate the greetings, insight, and doc link, thank you very much for including me in the forum! I did come across that doc earlier, unfortunately, I wasn’t able to adopt the primary/secondary Jace architecture due to limitations on how many IO modules are recommended to be hosted by a single Jace. The split control is not ideal but the idea is there is enough equipment control points split between the two Jaces and their individual IO modules to be able to satisfy requirements if either one of the Jaces goes down or all communication is lost.
I did adopt the direct local Niagara connection between the two Jaces using the secondary IP port and a single ethernet cable which, in theory, should keep the Jace to Jace communication reliable outside of someone with a pair of snips getting carried away inside the panel.
The randomized variable is a great idea that I hadn’t considered and I’ll be looking into testing that shortly. I did see the VykonPro module has a connection failover component but I think it could possibly run into the same lock up on timeouts issues like you mentioned, I will need to test that out to see how it behaves.
I wanted to comment on this post but held back because i wanted to find the document that outlines the process for a redundant JACE.
There’s a way to do it that involves Main and backup JACE plus a supervisory JACE that monitors the two. I used it on a job but for the life of me, I cannot find the document that outlined the process. (I’ve been looking for it for a month now)
Basic process was that the supervisor would detect main JACE being down and switch the primary IP of the backup JACE to be the same as main JACE. and then undo it once the main is back online. It’s a bit more nuanced but without the actual doc, this is the best i can recall.
The only downside to this that the backup JACE has to be updated regularly to ensure that it is up to date with any changes made on the main one. And this has to be done manually.
If i ever run across the doc, I will make sure to post it here.