In a resent YouTube video (Policy Center Architecture) I showed an example on how your Information Architecture can ensure that your structured content works with Search AND remains so flexible that additional requirements can be implemented without having to rethink the entire solution.
One of the considerations that we must address in any enterprise architecture is the lifecycle of the content. In the Policy Center scenario, we will have to deal with content that expires and thus becomes invalid.
I had three options to choose between:
"Do nothing", this option will as the name implies do nothing, the content remains available, and no warning is shown.
"Warning label" a warning is shown to the users, making sure they understand that the content might be invalid and act accordingly.
The third option, and the one we are going to address in this blog post, is "Hide" . By "Hide" we mean that the content must seem to disappear from the end user AND neither Search nor Copilot must be able to access the content.
As usually we have more than one way to achieve this goal:
Option "NoCrawl aka That would have been nice":
On Sites and Lists/Libraries we can set the "NoCrawl" property and ensure that the content in this container will not be crawled and thus not available in Search and Copilot.
If we could set "NoCrawl" on an Item/Document that would have been my preferred option, but alas, that is not possible. No such property exists at the item level. :-(
Option A: Permissions
By changing the item level permissions by breaking the inheritance and removing everybody but a few admin accounts (and the employee that should have reviewed the content), we have ensured that only a very few people can "see" the content.
Pro:
Works with the existing configuration, we don't have to add new lists or libraries.
Contra:
Item level permissions are no longer the huge issue it was back in the day, yet it is not recommended by Microsoft. When the List or Library contains more then 5000 items there are certain functions that no longer works, so I will recommend staying below that threshold.
This approach will require some additional handling IF the item already have customer permissions before we hide it, as we will have to persist the original permissions in order to reinstate them at a later date.
Option B: Move the content
Another approach is to create a clone of the Site Pages library "SitePagesHiddenFromSearch", and enable the "NoCrawl" property on this library. The library must use the same Content type as the Site Pages library.
Once content is marked as invalid, a workflow will MOVE the content to the hidden library.
One way to do this is to create two Logic Apps. This first one is triggered when a file is created or updated on the Site Pages Library.
The condition is that either the "isPolicyValid" is false or that the content has expired (ValidUntilDate less that utcNow() )
If either is true that call the Azure Function "MovePolicyToHiddenList", which will move the file to the hidden library.
(I know I could most likely could have set up the Logic App to do this without calling an Azure Function, but I LOVE my Azure Functions using PnP.PowerShell and a Managed Identity)
The second Logic App is doing almost the same, execpt the trigger is on the "SitePagesHiddenFromSearch", and the Move-PnPFile CMD is moving the file from the hidden library to the Site Pages library.
The final task is to create a job that moves content to the hidden list when the content expires. There are properbly many ways to achieve this, but in this case, I preferred an Azure Logic App triggered on a timer. This Logic App calls the same Azure Function as above if the content has expired.
I like the "move the content" approach, but it does have a severe problem: how to scale it and use it in a provisioning scenario where each new Site Collection will have this setup implemented on the Site Pages library.
I haven't found an elegant solution on this issue yet, but please reach out if you have a suggestion.