Unable to add reference in Visual Studio 2017 - Error HRESULT E_FAIL has been returned from a call to a COM component

While adding assembly reference in Visual Studio 2017, if you are facing the following error message: " Error HRESULT E_FAIL has been returned from a call to a COM component", this could be due to ReferenceManagerPackage failed to load. If you encountered this issue, here's a quick solution for you to resolve it.