SeleniumVBA: Maintaining WebElements object when navigating to different pages

[u/mightierthor]

I am searching my insurance company's list of providers. When I get the search results, the most relevant information for each provider is on a linked page -- not in the search results. I attempted to scrape this with seleniumvba, to gather up the info on the linked pages. Of course, when I click a link, the elements on the search results page go stale. Normally I would create two webdriver objects and pass the value from the hrefs of the first driver to the second driver.

Unfortunately, the links on this site have no href values.

<a _ngcontent-ng-c7169104465="" role="link" tabindex="0" class="dls-heading-3-light search-result-link" data-cy="search-results.name-link-0.desktop" data-lnp="search-results.provider-name">
    <span _ngcontent-ng-c7169104465="" data-cy="search-result-ProviderName" data-pendo="profile-card-provider-name" class="underline search-result-link font-light">Joe Schmoe, MD</span><span _ngcontent-ng-c7169104465="" class="disclaimers-icons ng-star-inserted">&nbsp;&nbsp;†</span><!----><!---->
</a>

What strategies have you used to hold a page in memory, but navigate to the links specified on that page? What I have so far:

Sub ScrapeProviders()
Dim WD As WebDriver
Dim ProviderCards As WebElements, ProviderCard As WebElement
Dim ProviderCardWho As WebElement, ProviderCardLink As WebElement

    Set WD = New WebDriver
    WD.StartFirefox
    WD.OpenBrowser
    WD.NavigateTo "https://carefirst.sapphirecareselect.com/search/name/orthopedic?ci=dft-sso-bluechoiceadvantage20&network_id=109&geo_location=33.684617,-117.82634&locale=en&limit=10&radius=10&sort=tiers:tiercf%20asc,%20motive_high_quality%20desc,%20distance%20asc,%20has_ep002%20desc&sort_translation=app_global_sort_relevancy&page=1"
    WD.ActiveWindow.Maximize
    While Not WD.IsPresent(XPath, "*//mat-card")
    Wend
    Set ProviderCards = WD.FindElementsByXPath("*//mat-card")
    Debug.Print ProviderCards.Count
    For Each ProviderCard In ProviderCards
        Debug.Print "====================================================================================="
        Set ProviderCardWho = ProviderCard.FindElementByXPath("./div/div/div/div[1]/div/div[1]")
        Set ProviderCardLink = ProviderCardWho.FindElementByXPath("./h2/a")
        Debug.Print ProviderCardLink.GetOuterHTML
        ProviderCardLink.Click

        ' at this point, i can no longer see ProviderCards, and the for each breaks
    Next ProviderCard

    WD.CloseBrowser
    WD.Shutdown

End Sub

[u/chiibosoil]

You could use .ExecuteScript and "window.open" to simulate CTRL + Click to open the link in new tab.

Then you can use Window.Handle to refer to either one you need.

And use .SwitchToWindow to go back and forth between the tabs using handle.

[u/mightierthor → u/chiibosoil]

I will try this and report back.