[RESOLVED] Server Outage - 7/18/2018

We are aware of an issue with a large number of client servers that has resulted in servers patching and rebooting outside of normal business hours. These issues appear to be a glitch in the server patching application and we're currently working on the problem. Our current estimated time for resolution is one hour but we will continue to update this channel as more updates are available.

Let us know if you have any questions by calling at 650-454-8850. 

12PM PST Update - The vast majority of servers should now be back online. Please give us a call at 650-454-8850 if you are still experiencing any issues, however this outage is pending clear.

11:15AM PST Update - Servers have started to come back online. We expect that bulk of all servers to be back up within the next 45 minutes, however a small subset of servers may require additional troubleshooting. We will reach out directly to any clients with servers that don't immediately come back up within this time frame.

[RESOLVED] Comcast Outage - 06/29/2018

At this time, it has come to our attention that Comcast is experiencing widespread  network connectivity issues throughout several US states. At this time, Comcast does not have an estimate for resolution, but we will pass on further updates via this webpage.

Please let us know if you have any questions by calling us at 650-454-8850.


Update 6/29 @ 2:25pm

Comcast tells CNBC that a "fiber cut" at a "large backbone network" partner has caused a service outage affecting its business and residential video, internet and voice customers.


Update 6/29 @ 3:12pm

Comcast provided an expected resolution time at 5:13pm PDT (Could possibly change due to the huge impact of the issue).


Update 6/29 @ 5:10pm

Comcast has been recovering all of their services and all networks should be back up. In case of specific issues, please do not hesitate to contact us.

Kinetix Support

Datacenter Major Outage

At this time we are experiencing an outage for some clients in our colo. This will affect RDS connectivity. We are investigating this further, and update here as new information is gained. Let us know if you have any questions by calling at 650-454-8850. 


Update 4/11 @ 8:52am PST

After investigation it appears that the source of this outage was related to a Microsoft update and has been manually address by Kinetix Support Staff. This issue is now pending close. We will continue to monitor and update this page once the outage has been cleared


Update 4/11 @ 10:07 am PST

After further investigation, it appears that the source of this outage was a Microsoft update that broke the network adapters for several virtual servers, forcing us to fix each adapter’s settings manually. This issue is now closed for all RDS users


Kinetix Support



[Resolved] Shortel Outage - 1/24

We've received multiple calls and discovered that the company Shortel is experiencing a major outage which is affecting all clients across the US/Canada. This is affecting entire phone systems as they are unable to connect to their services. We are continuing to reach out to Shortel for updates but as of now there is not an estimated time of return to service. Below is a link directly to Shortel's status page for more information. 


Update 5:15PM: 

After monitoring and calling clients back to verify, the outage has been cleared. Mitel (Formerly known as Shortel) has also cleared the outage on their website. If users are continuing to have issues with the connections on their phone please use the number below to reach out for support.

For any additional questions or assistance please reach out to the helpdesk at 650-454-8850. 


Kinetix Support

[Resolved] Major Outage - Multiple Providers

We are receiving calls from many of our clients experiencing issues across multiple internet providers. The issues appear to be related to some kind of routing issue between providers. We are investigating the issue and will be posting all updates here as well as hourly emails. 

Update 1:10PM:

Comcast has declared a mass outage that may be affecting all of their clients. Estimated time of recovery is 5:00PM.

We are still working with the other ISPs to gain information. If you are experiencing issues feel free to call the help desk at 650-454-8850 with any questions. 

Update 2:10PM

Comcast is still experiencing a mass outage. Estimated time of recovery has remained at 5:00pm. We will be moving all clients that have secondary circuits over to their backups.

Level 3 has confirmed that they experienced a nationwide outage at 9:00AM as well. However, this was cleared at about 12:00PM. 

We are currently still investigating the remaining ISPs to verify that the signals and outages are clear. If there are any questions or further issues please call the help desk at 650-454-8850 for assistance.

Update 4:22PM

After investigation it appears that the source of this outage was related to a nationwide DNS server issue that affected most, if not all providers. Service has been returned to many clients, and several ISPs like Level 3 and AT&T have cleared their outages.

While many clients of Comcast have service once more, they have extended the outage to 7:00PM. If service has still not returned, please call the help desk at 650-454-8850 for further support. We will continue to monitor and update this page once the Comcast outage has been cleared.

Update 8:43PM

After speaking with Comcast the outage has been officially cleared. While we were informed by a Comcast technician that the outage was related to a DNS server issue. We continued investigation and found an article related to today's outage. This article described the issue as a "configuration issue from Level 3". While the issue originated at the provider Level 3, it affected other vendors like Comcast due to Level 3's infrastructure which distributes for other services.

Service should now be returned to normal. If any issues remain please reach out to the help desk at 650-454-8550 for assistance. We thank you for your patience.


Kinetix Support

[RESOLVED] PG&E Power Outage - Menlo Park

At this time, there is a current power outage in the Menlo Park Area. It appears the power will be out for at least two hours. We are investigating this further, and update here as new information is gained. Let us know if you have any questions by calling at 650-454-8850. 

Update 9/28 @ 3:30pm PST

PG&E estimates the power will return around 5:30pm PST.

Update 9/28 @ 4:01pm PST

Power has returned.


Kinetix Support

[RESOLVED] 8/3 - Shoretel Major Outage

Hello Shoretel Supported Clients,

Shoretel is currently experiencing a major outage that affects most (if not all) customers. There is no ETA at this time, but we will keep checking each hour until the issue is resolved. Thank you for your patience during this downtime.

Update - 8/3 @ 9:55AM PST

The issue is still ongoing, but we will check on the status again around 11:00AM PST.

Update - 8/3 @ 11:42AM PST

The Shoretel outage has been resolved.

If you have any questions, please reach out to us at the helpdesk at 650-454-8850.


Kinetix Support

[RESOLVED] 8/1 Major Outage - Slack

Dear Slack Supported Clients, we have become aware that there is a major outage affecting Slack.  This affects our ability to respond to incoming Slack messages from our clients in real time.  We are working with Slack support to resolve this issue, and will provide an estimated time of repair as soon as it is known.  In the meantime, direct all urgent help requests to 650.454.8850, and non-urgent requests to help@kinetix.com.

Update - 8/2 @ 5:16PM PST

We have resolved the issues related to Slack messages. 


Kinetix Support

7/21 Helpdesk Phone System Outage - RESOLVED


Hello Kinetix Client,

It has come to our attention that our helpdesk phone system is experiencing an intermittent outage. When calling our 650-454-8850 number, you may be unable to get through. We are working on resolving this issue with our phone system provider at a high priority.

In the interim, please direct any urgent, business hours requests to our live chat service. You may find a link to our chat service service here - http://help.kinetix.com/


Kinetix Support



The issue with the Kinetix phone system has been resolved by the our phone provider. You may have your team resume opening cases, 24/7, at 650-454-8850





[RESOLVED] Egnyte Authentication issues - 7/3

Hello Kinetix Client,

It has been brought to our attention that Egnyte, a cloud hosted filesharing service, is currently undergoing issues prohibiting users from authenticating to their systems. Egnyte support is currently working on a process for remediation.

We will continue to stay in touch with Egnyte during the remediation process. For future updates on this issue, please visit Kinetix.com/status. If you have any questions, please give us a ring at 650-454-8850.

Below are all related updates from Egnyte:

At 7:13am PST- We are investigating an issue causing authentication issues with the Egnyte Drive app and Desktop Connect app. Please contact our support if you have experienced any issues.

At 7:55am PST - Customers from West coast region may be facing this issue. We are working on the mitigation process to bring our services back to operation status. We know some customer are facing login issues with their domain in the Web coast region. If you are experiencing any issues, please contact the Egnyte support team.

At 10:13am PST - We were able to identify the issues affecting Egnyte Drive services for some of our customer in the West coast. We have mitigated the issues and our EGD service is now operational. In the meantime we are monitoring the situation and our engineers are working on the root cause. We will continue to update EGD status and provide you with a postmortem report today.


Kinetix Support

Kinetix datacenter issue 6/27

We are aware of a momentary network issue associated with services hosted in the Kinetix datacenter and are investigating the root cause with our infrastructure provider. At this time all services are back online.

We will provide the root cause as soon as our investigation is complete.

If you have any questions or are noticing any further issues, please reach out to our helpdesk at 650-454-8850. We appreciate your understanding.

Major Outage - Slack

According to Slack support at 1:20pm PDT, "We are currently seeing a connectivity problem where some teams are unable to load Slack. We are very sorry for this disruption. We're working hard to solve it as quickly as we can. We will update this page with more information as it becomes available."

As of 1:38pm PDT: This issue is specifically affecting IBM, so this outage is considered resolved for all other users.

Please let us know if you have any questions by calling us at 650-454-8850.


Kinetix Support

Large Scale Spear Phishing Attack

UPDATE: Google has officially released a statement that they have stopped the attack.


Currently there is a large scale phishing attack that is propagating throughout the internet.  The attack appears as a legitimate email from an employee within your company or address list, and instructs the recipient to click on a link that looks like a legitimate link to a Google Document.  Please be aware and inform your employees to not click on this link.  If clicked, you are asked to grant permissions to an illegitimate Google Drive application that then parses your emails and contacts to send additional messages to. 

If you have clicked this link, please mitigate the attack by doing the following:

“[I]f you have clicked on the link, go to your Google account's page (https://myaccount.google.com/permissions) where you can manage the permissions you've granted to apps. Then locate the "Google Doc" app. This looks totally legitimate, but it's actually not. If that's the malicious app that's gotten access to your account after you clicked on the link it should have a recent "Authorization Time." Now, click on that Google Docs app and click Remove.”

More details can be found here: https://motherboard.vice.com/en_us/article/massive-gmail-google-doc-phishing-email

Please do not hesitate to reach out to our helpdesk directly at 650.454.8850 for assistance with this.


Kinetix Support

Power Outage in San Francisco

Hi Kinetix Clients,

We believe the PG&E Power outage is now resolved.

Issue: PG&E had a power outage affecting portions of San Francisco.

Root Cause: PG&E power outage was caused by an equipment failure at an electrical substation in the Tenderloin. It took them several hours to repair and this affected over 90,000 customers.

Please let us know if you have any questions by calling us at 650-454-8850.


Kinetix Support