Selenium Python getting <script> tag information?

Question

I am trying to take information inside the tag.

<script type="text/javascript"> INFO </script>

More specifically:

<!DOCTYPE html>
<html lang="en" class="js logged-in client-root">
<head>...</head>
<body class style ="position: fixed; top: -265px; width: 100%;">
  <span id="react-root" aria-hidden="true">...</span>
  <script type="text/javascript> This is the tag i want. </script>
  <script type="text/javascript> 
     window.__initialDataLoaded(window._sharedData); </script>
...

I am trying this but no luck:

browser = webdriver.Chrome()
info = browser.find_element_by_xpath("//script[@type='text/javascript']")
print(info.text) //prints nothing

can you post the URL of the website that you are trying or the HTML code? — Ajithkumar_sekar
– Ajithkumar_sekar, Commented May 18, 2018 at 14:25
instagram.com I am trying to extract comment times which are in <script> tag — doruksahin
– doruksahin, Commented May 18, 2018 at 14:31
@doruksahin Update the HTML with more of its preceding and following tags — undetected Selenium
– undetected Selenium, Commented May 18, 2018 at 14:39

Rolando Cruz · Accepted Answer · 2018-05-18 15:01:47Z

17

Seems like this is the expected behavior. See here:

This is working correctly. WebElement#getText returns the visible text that a user can see. The user cannot see text in a <script> tag, so it is not returned by getText. You can still access contents of the tag through JavaScript

So you will have to do something like this:

info.get_attribute('innerHTML')

answered May 18, 2018 at 15:01

Rolando Cruz

2,7841 gold badge17 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

doruksahin Over a year ago

Thanks a lot. By the way, I get page source and inspector from my browser. When i compare them something is different. Page source has commented time information but inspecting won't help me.

Rolando Cruz Over a year ago

Sorry but can you please elaborate more on what you mean by "commented time information"?

doruksahin Over a year ago

I am scraping instagram. When im using Chrome manually, checked "Page source" and "Inspect element". I compared them and they wasnt the same. driver.page_source gave me "Inspect element" info, not "Page source". Is there a way i can scrap "Page source" with selenium?

Rolando Cruz Over a year ago

If you want page source, then a scraper might be a better option. Try looking at BeautifulSoup (crummy.com/software/BeautifulSoup/bs4/doc)

Collectives™ on Stack Overflow

Selenium Python getting <script> tag information?

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related