Otava Status Page Status

Monitoring - A fix has been implemented and we are monitoring the results.
Feb 4, 16:16 EST

Identified - Otava is aware of the remote code execution vulnerability in Apache Log4j. This vulnerability is also known as Log4shell and has the CVE assignment CVE-2021-44228

Background:
Log4j is a Java based logging audit framework. Log4j2 2.14.1 and below are susceptible to a remote code execution vulnerability where a remote attacker can leverage this vulnerability to take full control of a vulnerable machine.

IMPORTANT: This is a notice of a critical vulnerability and related threat; this is NOT a notice that we have been compromised as a result of this vulnerability.

Otava Security has been actively working with our third-party service partners and will apply critical patches immediately upon release. Otava has currently applied "known" security signatures for perimeter defense sensors, is actively scanning for such security vulnerabilities in all data centers and will continue active monitoring for all data centers for any bad actors and related malicious behavior.

If Otava does not manage your infrastructure devices we still recommend you follow vendor best practices and guidelines regarding patches for this vulnerability.

If you have questions about products or services that Otava can help provide please reach out to your account representative for further discussion.

Status updates can be viewed at Status.Otava.com where this incident is published. ALL customers can subscribe and receive real-time updates as we proceed.

Thank you,

Otava Operations
Dec 14, 19:01 EST

Investigating - VMware has released a notification of a critical vulnerability related to Apache. Otava is currently investigating the vulnerability and will be applying changes within the Otava network infrastructure in the attempt to mitigate any risk while Otava works with the vendor.

CVE: CVE-2021-44228
Dec 12, 16:17 EST

AU1 - Austin, TX Operational

Public Cloud Operational

Private Cloud Operational

Next Gen Cloud (Gen3) Operational

Networking Operational

Security Operational

Otava Cloud Backup (OCB) Operational

Otava Cloud Connect (OCC) Operational

Otava Cloud Backup for MS365 (OCB-MS365) Operational

DRaaS Operational

Data Center Facilities Operational

NV1 - Las Vegas, NV Operational

Public Cloud Operational

Private Cloud Operational

Next Gen Cloud (Gen3) Operational

Networking Operational

Security Operational

Otava Cloud Backup (OCB) Operational

Otava Cloud Connect (OCC) Operational

Otava Cloud Backup for MS365 (OCB-MS365) Operational

DRaaS Operational

Data Center Facilities Operational

IN1 - Indianapolis, IN Operational

Public Cloud Operational

Private Cloud Operational

Next Gen Cloud (Gen3) Operational

Networking Operational

Security Operational

Monitoring (OTMonitor) Operational

Otava Cloud Backup (OCB) Operational

Otava Cloud Connect (OCC) Operational

Otava Cloud Backup for MS365 (OCB-MS365) Operational

DPaaS Operational

DRaaS Operational

Data Center Facilities Operational

MM1 - Flint, MI (Mid-Michigan) Operational

Public Cloud Operational

Private Cloud Operational

Networking Operational

Security Operational

Monitoring (OTMonitor) Operational

Otava Cloud Backup (OCB) Operational

Otava Cloud Connect (OCC) Operational

Otava Cloud Backup for MS365 (OCB-MS365) Operational

DPaaS Operational

DRaaS Operational

Data Center Facilities Operational

Networking Operational

MD1 - Metro Detroit, MI Operational

Private Cloud Operational

Networking Operational

Security Operational

Monitoring (OTMonitor) Operational

Otava Cloud Backup for MS365 (OCB-MS365) Operational

DPaaS Operational

DRaaS Operational

Data Center Facilities Operational

AA2 - Ann Arbor, MI Operational

Private Cloud Operational

Networking Operational

Security Operational

Monitoring (OTMonitor) Operational

DPaaS Operational

DRaaS Operational

Data Center Facilities Operational

VA4 - Ashburn, VA Operational

Public Cloud Operational

Private Cloud Operational

Networking Operational

Security Operational

Data Center Facilities Operational

MA3 - Manchester, UK Operational

Public Cloud Operational

Private Cloud Operational

Networking Operational

Security Operational

Otava Cloud Connect (OCC) Operational

Data Center Facilities Operational

DEN1 - Denver, CO Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMWare Operational

Data Center Facilities Operational

NYC1 - New York, NY Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

DAL1 - Dallas, TX Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

PHX1 - Phoenix, AZ Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

CLE1 - Cleveland, OH Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

DUS1 - Dusseldorf, Germany Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

FRA1 - Frankfort, Germany Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

LA1 - Los Angeles, CA Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

SF1 - San Fransico, CA Operational

vCloud Operational

Networking Infrastructure Operational

Internet Backbone Operational

Security Operational

Otava Cloud Connect (OCC) Operational

DRaaS powered by Veeam Operational

DRaaS powered by VMware Operational

Data Center Facilities Operational

Client Services Operational

OTPortal (portal.otava.com) Operational

Partner Portal (partner.otava.com) Operational

Support Portal (support.otava.com) Operational

Website (otava.com) Operational

VMware Horizon View (apps.otava.com) Operational

Phone System Operational

Operational

Degraded Performance

Partial Outage

Major Outage

Maintenance

Scheduled Maintenance

Austin(AU1), Las Vegas(NV1), Indianapolis(IN1), Genesee County(MM1), ASHBURN (VA4) - ZVM Patching & Restart May 30, 02:00-08:00 EDT

Maintenance: Zerto Virtual Manager (ZVM) Patching & Restart

Maintenance Classification: Scheduled

Start Time: May 30, 2022 at 02:00 AM ET.

End Time: May 30, 2022 at 08:00 AM ET.

Type: Scheduled maintenance

Service Impacting: Yes

Description: Otava will be performing a scheduled patching & restart to the Zerto Virtual Manager (ZVM) server.

This notification is establishing a regular scheduled patching and the restart of the ZVM.

During the period of the maintenance the following will be true:

· Replication for your protected systems will NOT be impacted.

· Partial access to the Zerto user interface will not be available during this time period, however, because we will only be performing the maintenance on one ZVM at a time, an emergency DR failover is still possible.

· If during this time you need assistance with a DR failover, please contact support.

Estimated Downtime: During the maintenance period access to the Zerto user interface will be unavailable for an estimated 3 Hours. Replication will NOT be impacted by this event.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
Posted on May 2, 03:52 EDT

Past Incidents

May 27, 2022

Austin (AU1) to Ashburn (VA4) Transport Circuit ISP Maintenance - 5/27

Completed - The scheduled maintenance has been completed.
May 27, 06:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 27, 01:00 EDT

Scheduled - Start Time: 5/27 - 1:00 AM EDT

End Time: 5/27 - 6:00 AM EDT

Type: ISP network optimization

Service Impacting: Yes

Description: The ISP is performing network optimization.

Estimated Downtime: Up to 2 seconds as the circuit gracefully fails over to the redundant circuit.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 17, 16:59 EDT

May 26, 2022

Austin (AU1) to Ashburn (VA4) Transport Circuit ISP Maintenance - 5/26

Completed - The scheduled maintenance has been completed.
May 26, 06:01 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 26, 01:00 EDT

Scheduled - Start Time: 5/26 - 1:00 AM EDT

End Time: 5/26 - 6:00 AM EDT

Type: ISP network optimization

Service Impacting: Yes

Description: The ISP is performing network optimization.

Estimated Downtime: Up to 2 seconds as the circuit gracefully fails over to the redundant circuit.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 17, 16:55 EDT

Mid-Michigan (MM) Infrastructure Firewall Maintenance - 5/25 - 5/26 EXTENDED

Completed - The scheduled maintenance has been completed.
May 26, 04:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 26, 02:00 EDT

Scheduled - Start Time: 5/25 - 11:00 PM EDT

End Time: 5/26 - 02:00 AM EDT

Type: Firewall Firmware Upgrade

Service Impacting: Yes

Description: Otava network engineers will be upgrading the Infrastructure firewalls at MM. The upgrade is a three-step code upgrade that will cause the firewall in the HA stack to fail-over three times during the upgrades resulting in a few moments of packet-loss. DPaaS jobs could be interrupted during the upgrade, but should restart once complete.

Estimated Downtime: Three periods of up to 5 seconds of packet loss while firewalls fail-over.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 26, 00:48 EDT

Mid-Michigan (MM) Infrastructure Firewall Maintenance - 5/25

Completed - The scheduled maintenance has been completed.
May 26, 00:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 25, 23:00 EDT

Scheduled - Start Time: 5/25 - 11:00 PM EDT

End Time: 5/25 - 11:59 PM EDT

Type: Firewall Firmware Upgrade

Service Impacting: Yes

Description: Otava network engineers will be upgrading the Infrastructure firewalls at MM. The upgrade is a three-step code upgrade that will cause the firewall in the HA stack to fail-over three times during the upgrades resulting in a few moments of packet-loss. DPaaS jobs could be interrupted during the upgrade, but should restart once complete.

Estimated Downtime: Three periods of up to 5 seconds of packet loss while firewalls fail-over.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 23, 08:45 EDT

May 25, 2022

Otava Cloud Connect Infrastructure Patching

Update - The scheduled maintenance has been completed.
May 25, 12:31 EDT

Completed - The scheduled maintenance has been completed.
May 25, 12:31 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 25, 06:30 EDT

Scheduled - Otava engineers are establishing a maintenance window to perform patching on our Otava Cloud Connect infrastructure at our following datacenter locations :
AU1-Austin
NV1-Las Vegas
MA3-Manchester
IN1-Indianapolis
MM1-Flint
These patches are required in order to continue to provide a secure and compliant environment.

Service Impacting: Yes - Running backup jobs may fail or be cancelled, new jobs may not start until all services are upgraded and maintenance is complete.

We will kindly ask you to check your backup jobs and restart them if needed once the maintenance window is complete.

Estimated Downtime: Intermittently throughout the maintenance window

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 2, 03:50 EDT

May 24, 2022

Metro Detroit (MD1) Data Center Annual UPS Preventative Maintenance 5/24/22

Completed - The scheduled maintenance has been completed.
May 24, 17:01 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 24, 08:00 EDT

Scheduled - Start Time: 5/24/22 - 8:00 AM EDT

End Time: 5/24/22 - 5:00 PM EDT

Type: UPS Annual Preventative Maintenance

Service Impacting: Low

Description: Otava engineers, working with vendor engineers, are scheduling a maintenance window to perform preventative maintenance on the Uninterruptible Power Supply.

Estimated Downtime: Due to the redundant nature of the design, no impact on power is expected.
If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

MD1 - Metro Detroit, MI - Building Systems
May 12, 08:59 EDT

May 23, 2022

No incidents reported.

May 22, 2022

No incidents reported.

May 21, 2022

Mid-Michigan (MM) Storage Upgrades - 5/21

Completed - The scheduled maintenance has been completed.
May 21, 16:01 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 21, 14:00 EDT

Scheduled - Start Time: 5/21 - 2:00 PM EDT

End Time: 5/21 - 4:00 PM EDT

Type: Firmware Update

Service Impacting: Low

Description: Otava and Vendor engineers are performing firmware updates on storage systems for added resiliency and security.

Estimated Downtime: No impact expected.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 16, 14:37 EDT

Mid-Michigan (MM) Tier 2 Storage Upgrades - 5/21

Completed - The scheduled maintenance has been completed.
May 21, 16:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 21, 14:00 EDT

Scheduled - Start Time: 5/21 - 2:00 PM EDT

End Time: 5/21 - 4:00 PM EDT

Type: Firmware Update

Service Impacting: Low

Description: Otava and Vendor engineers are performing firmware updates on tier 2 storage systems for added resiliency and security.

Estimated Downtime: No impact expected.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 13, 16:15 EDT

May 20, 2022

No incidents reported.

May 19, 2022

AU - sCloud vCenter (AU-PubVC3) inaccessible

Resolved - This incident has been resolved.
May 19, 20:13 EDT

Monitoring - vCenter is back up and we're continuing to monitor. Please reach out to Otava support if you have any questions.
May 17, 23:47 EDT

Investigating - Otava Engineers are currently investigating an outage of the AU-PubVC3 vCenter. Running VMs are not affected as vCenter only provides management functions. We will update you as we have more information.
May 17, 23:09 EDT

Urgent Indianapolis (IN1) Switch Replacement - 5/19

Completed - The scheduled maintenance has been completed.
May 19, 20:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 19, 14:01 EDT

Scheduled - Start Time: 5/19 - 2:00 PM EDT

End Time: 5/19 - 8:00 PM EDT

Type: Urgent switch replacement

Service Impacting: Low

Description: Otava engineers will be replacing a switch in the Gen3 environment.

Estimated Downtime: No downtime expected.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 18, 16:26 EDT

Urgent Austin (AU1) vCenter Certificate Troubleshooting - 5/19

Completed - The scheduled maintenance has been completed.
May 19, 18:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 19, 15:00 EDT

Scheduled - Start Time: 5/19 - 3:00 PM EDT

End Time: 5/19 - 6:00 PM EDT

Type: Urgent vCenter STS Certificate Troubleshooting

Service Impacting: Low

Description: Otava engineers will be troubleshooting vCenter STS Certificates in sCloud AU-PubVC3.

Estimated Downtime: Maintenance should not be impactful however, VM management could be interrupted during the vCenter reboot.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 18, 12:03 EDT

Ann Arbor 2 (AA2) Data Center Electrical Systems Generator Minor Inspection 5/19/22

Completed - The scheduled maintenance has been completed.
May 19, 17:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 19, 08:02 EDT

Scheduled - Start Time: 5/19/22 - 8:00 AM EDT

End Time: 5/19/22 - 5:00 PM EDT

Type: Generator Preventative Maintenance

Service Impacting: Low

Description: Otava and vendor engineers will be performing routine annual minor maintenance inspections on the backup generators at AA2. We do not anticipate any impact as a result of this work, in the spirit of transparency we wanted to make you aware of the maintenance. Due to redundant power systems, no impact is expected.

Estimated Downtime: No impact expected If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

AA2 - Ann Arbor, MI - Building Systems
May 5, 13:20 EDT

Mid-Michigan Data Center Annual Major Generator Inspection

Completed - The scheduled maintenance has been completed.
May 19, 08:01 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 17, 08:01 EDT

Update - We will be undergoing scheduled maintenance during this time.
May 11, 12:59 EDT

Scheduled - Mid-Michigan Data Center Annual Major Generator Inspection

Start Time: 5/17/22 - 8:00 AM EDT

End Time: 5/18/22 - 5:00 PM EDT

Type: Annual Major Generator Inspection and Load Bank Testing

Description: Otava and vendor engineers will be performing routine maintenance on the backup generators at the Mid Michigan data center. During the maintenance, Otava will load test each generator one at a time and perform preventative maintenance inspections. We do not anticipate any impact as a result of this work, in the spirit of transparency we wanted to make you aware of the maintenance. Due to redundant power systems, no impact is expected.

Estimated Downtime: No impact expected If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

MM1 - Genesee County, MI - Data Center Facilities
May 11, 12:57 EDT

May 18, 2022

Ann Arbor (AA2) Utility Power Outage 5/18/22

Resolved - Start Time: 5/18/22 - 4:08 PM EDT

End Time: 5/18/22 - 6:25 PM EDT

Type: Utility Power Outage

Service Impacting: Low

Description: DTE reported equipment failure upstream from the data center.

Estimated Downtime: Due to the redundant nature of the data center design no impact.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 18, 18:37 EDT

Otava Cloud Backup Infrastructure Patching

Completed - The scheduled maintenance has been completed.
May 18, 12:30 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 18, 06:31 EDT

Scheduled - Otava engineers are establishing a maintenance window to perform patching on our Otava Cloud Backup infrastructure at our following datacenter locations :
AU1-Austin
NV1-Las Vegas
IN1-Indianapolis
MM1-Flint
These patches are required in order to continue to provide a secure and compliant environment.

Service Impacting: Yes - Running backup jobs may fail or be cancelled, new jobs may not start until all services are upgraded and maintenance is complete.

We will kindly ask you to check your backup jobs and restart them if needed once the maintenance window is complete.

Estimated Downtime: Intermittently throughout the maintenance window

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 2, 03:49 EDT

May 17, 2022

Mid Michigan (MM) DPaaS Firewall Maintenance - 5/17

Completed - The scheduled maintenance has been completed.
May 17, 15:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 17, 14:00 EDT

Scheduled - Start Time: 5/17 - 2:00 PM EDT

End Time: 5/17 - 3:00 PM EDT

Type: Firewall lifecycle management

Service Impacting: Yes

Description: Otava engineers will be performing lifecycle maintenance on the DPaaS firewalls.

Estimated Downtime: Otava will be unable to perform restores during the maintenance window. Also, clients may experience backup failures during the maintenance window. For any failed backup jobs, the Otava Operations team will run on-demand backup jobs after the maintenance to maintain backup retention.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 10, 16:28 EDT

Indianapolis (IN1) DPaaS Firewall Maintenance - 5/17

Completed - The scheduled maintenance has been completed.
May 17, 14:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 17, 13:00 EDT

Scheduled - Start Time: 5/17 - 1:00 PM EDT

End Time: 5/17 - 2:00 PM EDT

Type: Firewall lifecycle management

Service Impacting: Yes

Description: Otava engineers will be performing lifecycle maintenance on the DPaaS firewalls.

Estimated Downtime: Otava will be unable to perform restores during the maintenance window. Also, clients may experience backup failures during the maintenance window. For any failed backup jobs, the Otava Operations team will run on-demand backup jobs after the maintenance to maintain backup retention.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 10, 16:26 EDT

May 16, 2022

Urgent Austin (AU1) and Las Vegas (NV1) vCenter Certificate Maintenance - 5/16

Completed - The scheduled maintenance has been completed.
May 16, 19:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 16, 18:02 EDT

Scheduled - Start Time: 5/16 - 6:00 PM EDT

End Time: 5/16 - 7:00 PM EDT

Type: Urgent vCenter STS Certificate Update

Service Impacting: Low

Description: Otava engineers will be updating the vCenter STS Certificates in sClouds AU-PubVC3 and LV-PubVC3.

Estimated Downtime: Maintenance should not be impactful however, VM management could be interrupted during the vCenter reboot.

If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.
May 16, 13:29 EDT

Metro Detroit (MD1) Data Center Electrical System Major Generator Inspection and Load Bank Testing 5/16/22

Completed - The scheduled maintenance has been completed.
May 16, 17:00 EDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
May 16, 08:00 EDT

Scheduled - Start Time: 5/16/22 - 8:00 AM EDT

End Time: 5/16/22 - 5:00 PM EDT

Type: Annual Major Generator Inspection and Load Bank Testing

Description: Otava and vendor engineers will be performing routine maintenance on the backup generators at MD1. During the maintenance, Otava will load test each generator one at a time and perform preventative maintenance inspections. We do not anticipate any impact as a result of this work, in the spirit of transparency we wanted to make you aware of the maintenance. Due to redundant power systems, no impact is expected.

Estimated Downtime: No impact expected If you have any questions or concerns regarding this maintenance please contact us by logging into https://support.otava.com or calling 877-740-5028, option 3.

MD1 - Metro Detroit, MI - Building Systems
May 2, 15:45 EDT

May 15, 2022

No incidents reported.

May 14, 2022

No incidents reported.

May 13, 2022

No incidents reported.