tag:status.datagrail.io,2005:/historyDataGrail Status - Incident History2024-03-28T20:55:21-07:00DataGrailtag:status.datagrail.io,2005:Incident/167748442023-04-06T18:59:03-07:002023-04-06T18:59:03-07:00Trouble logging into DataGrail via Microsoft SSO<p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>18:59</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved and users with Microsoft SSO can now login to DataGrail successfully.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>16:37</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with logging into DataGrail via Microsoft SSO only.<br /><br />We are actively investigating and will post updates as we learn more.</p>tag:status.datagrail.io,2005:Incident/118481062022-10-11T16:42:11-07:002022-10-11T16:42:11-07:00Direct Contact Integrations Closing Requests without Completion after 14 Day Due Date<p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>16:42</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Oct <var data-var='date'>11</var>, <var data-var='time'>14:29</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with Direct Contact Integrations that have reached their 14-day expiration date and are set to "Contact My Team".<br /><br />Intended Behavior : At the 14 day expiration date, if the Direct Contact integration has not been completed by the designated email processor and is set to "Contact My Team", the Super Admin will get a notification to reach out to the designated email processor and the request will remain open until either the Request Processor/Super Admin manually stops processing OR the designated email processor for the Direct Contact integration completes the task.<br /><br />Current Behavior : At the 14 expiration date, if the Direct Contact integration has not been completed by the designated email processor and is set to "Contact My Team", the Super Admin is getting a notification still but the Direct Contact integration is marking itself as complete without the influence of the Request Processor/Super Admin or the task being actioned by the designated email processor, thus causing the request to close when all other integrations have completed.<br /><br />We are actively investigating and will post updates as we learn more.</p>tag:status.datagrail.io,2005:Incident/113660962022-10-03T14:47:32-07:002022-10-03T14:47:32-07:00Direct Contact Integrations Not Initiating Email Workflow<p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>14:47</var> PDT</small><br><strong>Resolved</strong> - This issue has now been resolved with very minimal customer impact. We are working with those impacted and have rectified the issue.<br /><br />Please reach out to support@datagrail.io with any questions!</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>13:02</var> PDT</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Oct <var data-var='date'> 3</var>, <var data-var='time'>13:02</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with Direct Contact integrations that were newly connected on or after 09/28, not initiating the email workflow of sending requests for completion by designated Email Processors.<br /><br />We are actively investigating and will post updates as we learn more.</p>tag:status.datagrail.io,2005:Incident/110289432022-08-31T19:28:15-07:002022-08-31T19:28:15-07:00Hybrid Integrations Not Sending Direct Contact Emails to Processors<p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>19:28</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>15:41</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with Hybrid integrations not sending Direct Contact emails to Processors. This is the result of the new Authenticated DSR Handling feature release from 08/30 that has since been reverted.<br /><br />We have identified the root cause and are working to fix and rerelease the feature. In the meantime, we are also actively working on resending all Direct Contact emails that should have gone out within the last ~24hrs and were impacted.<br /><br />We will update once the issue is fully resolved.</p>tag:status.datagrail.io,2005:Incident/98995562022-05-03T13:30:24-07:002022-05-03T13:30:24-07:00DataGrail Scheduled Maintenance<p><small>May <var data-var='date'> 3</var>, <var data-var='time'>13:30</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 3</var>, <var data-var='time'>12:30</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 2</var>, <var data-var='time'>17:13</var> PDT</small><br><strong>Scheduled</strong> - DataGrail will be performing scheduled maintenance to update our secrets encryption layer for continued optimal security on Tuesday, May 3rd at 12:30pm PST. The maintenance period will last for 1 hour and will not impact your ability to operate within our platform. Please reach out to support@datagtail.io should you experience any issues with your API integrations during this time.</p>tag:status.datagrail.io,2005:Incident/98682632022-04-27T20:15:23-07:002022-04-27T20:15:23-07:00DataGrail Scheduled Maintenance, weekday month/date, time timezone<p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>20:15</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>19:45</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>17:16</var> PDT</small><br><strong>Scheduled</strong> - DataGrail will be performing scheduled maintenance today, April 27th at 7:45pm PST. The maintenance period will last for up to 30min. The DataGrail platform will be unavailable during this time. All normal operations on the platform will resume immediately after the maintenance window.</p>tag:status.datagrail.io,2005:Incident/92943322022-02-11T13:12:51-08:002022-02-11T13:12:51-08:00DataGrail Account Invites resulting in 500 errors for Customers that utilize GoogleSSO or MicrosoftSSO<p><small>Feb <var data-var='date'>11</var>, <var data-var='time'>13:12</var> PST</small><br><strong>Resolved</strong> - This incident has been fully resolved and the platform is functioning normally.</p><p><small>Feb <var data-var='date'>11</var>, <var data-var='time'>12:22</var> PST</small><br><strong>Identified</strong> - We have identified an issue with DataGrail Account Invites leading to 500 errors for Customers that utilize GoogleSSO or MicrosoftSSO to log into DataGrail. <br /><br />We have identified a fix and are currently validating this internally.</p>tag:status.datagrail.io,2005:Incident/92844152022-02-10T11:00:53-08:002022-02-11T12:23:07-08:00Data Subject Requesters Unable to Verify Email Address<p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>11:00</var> PST</small><br><strong>Resolved</strong> - This incident has been fully resolved and the platform is functioning normally.</p><p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>09:21</var> PST</small><br><strong>Monitoring</strong> - We have resolved the issue with data subject requesters hitting an error when attempting to verify their email address and we are currently monitoring for any issues.</p><p><small>Feb <var data-var='date'>10</var>, <var data-var='time'>08:18</var> PST</small><br><strong>Identified</strong> - We have identified an issue with data subject requesters hitting an error when attempting to verify their email address.<br /><br />We are actively investigating and will post updates as we learn more.</p>tag:status.datagrail.io,2005:Incident/88557682021-12-15T08:43:13-08:002021-12-15T08:43:13-08:00Network Connectivity Issues<p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>08:43</var> PST</small><br><strong>Resolved</strong> - This incident has been fully resolved and the platform is functioning normally.</p><p><small>Dec <var data-var='date'>15</var>, <var data-var='time'>08:01</var> PST</small><br><strong>Identified</strong> - Users may experience intermittent issues connecting and logging into the DataGrail platform. DataGrail is working with our partners to resolve this issue.</p>tag:status.datagrail.io,2005:Incident/87359532021-12-01T12:20:25-08:002021-12-01T12:20:48-08:00Live Data Map: Multiple Inventory System Reports Not Showing<p><small>Dec <var data-var='date'> 1</var>, <var data-var='time'>12:20</var> PST</small><br><strong>Resolved</strong> - There is no data loss associated with this incident and all Multiple System Inventory Reports should be appearing as previously created. This incident existed for 28 minutes from 11:50am to 12:18pm PST on 12/1/21. <br /><br />This incident has been fully resolved and the platform is functioning normally.</p>tag:status.datagrail.io,2005:Incident/83532172021-10-30T23:01:30-07:002021-10-30T23:01:30-07:00DataGrail Scheduled Maintenance, Saturday October 30, 10pm PST<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>23:01</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>22:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>29</var>, <var data-var='time'>15:42</var> PDT</small><br><strong>Scheduled</strong> - DataGrail will be performing scheduled maintenance on Saturday, October 30 at 10pm PST. The maintenance period will last for one hour. The DataGrail platform will be unavailable during this time. All normal operations on the platform will resume immediately after the maintenance window.</p>tag:status.datagrail.io,2005:Incident/78848612021-08-31T17:03:38-07:002021-08-31T17:03:38-07:00Network Connectivity Issues<p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>17:03</var> PDT</small><br><strong>Resolved</strong> - This incident has been fully resolved and the platform is functioning normally.</p><p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>16:10</var> PDT</small><br><strong>Monitoring</strong> - Issues with intermittent connectivity with the DataGrail platform have been resolved and we are currently monitoring this fix.</p><p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>12:07</var> PDT</small><br><strong>Update</strong> - Users may experience intermittent issues connecting and logging into the DataGrail platform. DataGrail is working with our partners to resolve this issue.</p><p><small>Aug <var data-var='date'>31</var>, <var data-var='time'>11:48</var> PDT</small><br><strong>Identified</strong> - Users may experience intermittent issues connecting and logging into the DataGrail platform. DataGrail is working with our partners to resolve this issue.</p>tag:status.datagrail.io,2005:Incident/78454902021-08-25T21:16:56-07:002021-08-25T21:16:56-07:00Issue with Direct Contact Deletions (Resolved)<p><small>Aug <var data-var='date'>25</var>, <var data-var='time'>21:16</var> PDT</small><br><strong>Resolved</strong> - Today (8/25/2021) we identified and resolved an issue with Deletions in which privacy requests were automatically closed before a response was recorded from Direct Contact Integrations. <br /><br />We have reset all affected privacy requests back to a “Pending Delete” state. No data or requests were lost. Please review your queue and process requests as you would normally.<br /><br />If you have any questions please contact support@datagrail.io.</p>tag:status.datagrail.io,2005:Incident/75098262021-07-16T19:30:15-07:002021-07-16T19:30:15-07:00DataGrail Scheduled Maintenance, Friday July 16, 7pm PST<p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>19:30</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance was completed successfully. There was no downtime impact on the DataGrail Platform.</p><p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>19:28</var> PDT</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Jul <var data-var='date'>16</var>, <var data-var='time'>19:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jul <var data-var='date'>14</var>, <var data-var='time'>18:37</var> PDT</small><br><strong>Scheduled</strong> - DataGrail will be performing scheduled maintenance on Friday, July 16 at 7pm PST. The maintenance period will last for one hour. The DataGrail platform will be unavailable during this time. All normal operations on the platform will resume immediately after the maintenance window.</p>tag:status.datagrail.io,2005:Incident/72219982021-06-10T20:58:58-07:002021-06-15T11:11:16-07:00Service disruption during routine database upgrade<p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>20:58</var> PDT</small><br><strong>Resolved</strong> - The incident has been resolved. Please contact support@datagrail.io if you have questions or would like a copy of our root cause analysis (once available).</p><p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>20:45</var> PDT</small><br><strong>Monitoring</strong> - The service disruption has been resolved and all systems are back online. We are actively monitoring.</p><p><small>Jun <var data-var='date'>10</var>, <var data-var='time'>20:30</var> PDT</small><br><strong>Identified</strong> - A service disruption started around 8:11 PM PT during a routine database upgrade. We are actively resolving the issue and should be back up shortly. Updates to follow.</p>tag:status.datagrail.io,2005:Incident/71451202021-06-01T12:00:00-07:002021-06-02T00:06:30-07:00Privacy Intake Forms Loading Inconsistently<p><small>Jun <var data-var='date'> 1</var>, <var data-var='time'>12:00</var> PDT</small><br><strong>Resolved</strong> - DataGrail's Engineering Team were alerted by internal monitoring and proactively remedied a memory issue that was appearing sporadically across privacy intake forms, impacting the ability to load these forms.The issue was discovered at 11:50am PST and resolved at 12:05pm PST.<br /><br />This incident has been fully resolved and the platform is functioning normally.</p>tag:status.datagrail.io,2005:Incident/69291472021-05-05T14:13:22-07:002021-05-05T14:13:22-07:00Issue Detected with Smart Verification<p><small>May <var data-var='date'> 5</var>, <var data-var='time'>14:13</var> PDT</small><br><strong>Resolved</strong> - This incident has been fully resolved and the platform is functioning normally.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>14:03</var> PDT</small><br><strong>Monitoring</strong> - We have resolved the issue with Smart Verification and we are currently monitoring for any issues.<br /><br />We have resent verification emails to all requesters that encountered an error when attempting to verify their privacy request.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>13:29</var> PDT</small><br><strong>Update</strong> - We are continuing work on a fix for this and will post updates as we learn more.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>12:51</var> PDT</small><br><strong>Update</strong> - We are actively working on a fix for this and will post updates as we learn more.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>12:23</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with certain flows for our Smart Verification service. We will post updates as soon as more information is available.</p>tag:status.datagrail.io,2005:Incident/69073562021-05-03T13:50:30-07:002021-05-03T15:04:30-07:00Access Result Files being sent with partial information<p><small>May <var data-var='date'> 3</var>, <var data-var='time'>13:50</var> PDT</small><br><strong>Resolved</strong> - This incident has been fully resolved and the platform is functioning normally.</p><p><small>May <var data-var='date'> 3</var>, <var data-var='time'>13:25</var> PDT</small><br><strong>Monitoring</strong> - We have resolved the issue with access result files being sent with partial information and we are currently monitoring.<br /><br />We have re-sent all affected access result emails to the appropriate consumers.</p><p><small>May <var data-var='date'> 3</var>, <var data-var='time'>12:58</var> PDT</small><br><strong>Identified</strong> - We have identified an issue with access result files being sent with partial information.<br /><br />We are actively investigating and will post updates as we learn more.</p>tag:status.datagrail.io,2005:Incident/67155192021-04-09T12:36:54-07:002021-04-09T12:36:54-07:00CAPTCHA Failing to Load On Intake Forms<p><small>Apr <var data-var='date'> 9</var>, <var data-var='time'>12:36</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved and the platform is functioning normally.</p><p><small>Apr <var data-var='date'> 9</var>, <var data-var='time'>10:57</var> PDT</small><br><strong>Monitoring</strong> - DataGrail's Engineering team has resolved the issue with CAPTCHAs not loading and we are currently monitoring for any issues.</p><p><small>Apr <var data-var='date'> 9</var>, <var data-var='time'>10:38</var> PDT</small><br><strong>Identified</strong> - DataGrail's Engineering team has identified the cause of CAPTCHAs failing to load on customer intake forms. The team is currently working on resolving this issue. We will provide another update in 15 minutes.</p>tag:status.datagrail.io,2005:Incident/65457622021-03-16T14:09:05-07:002021-03-16T14:09:30-07:00Emails not sending<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>14:09</var> PDT</small><br><strong>Resolved</strong> - We have fixed this issue as of now. Emails should be sending as expected.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>10:54</var> PDT</small><br><strong>Monitoring</strong> - Impact has been resolved. Long term fix is ongoing</p>