python - Removing newlines (\n) with BeautifulSoup -


i'm parsing html page bs4:

import re import codecs import mysqldb bs4 import beautifulsoup  soup = beautifulsoup(open("sprt.htm"), from_encoding='utf-8') sprt = [[0 x in range(3)] x in range(300)] = 0  para in soup.find_all('p'):     if para.strong not none:         sprt[i][0] = para.strong.get_text()         sprt[i][1] = para.get_text()         sprt[i][1] = re.sub(re.escape(sprt[i][0]), "", sprt[i][1], re.unicode)         sprt[i][2] = sprt[i][1]         sprt[i][2] = re.sub(r".+[\.\?][\s\s\n]", "", sprt[i][1], re.s)         sprt[i][2] = re.sub(r".+panel", "panel", sprt[i][2], re.s)         sprt[i][1] = re.sub(re.escape(sprt[i][2]), "", sprt[i][1])  += 1 x = 0

the page i'm parsing filled paragraphs 3:

<p><strong>name name. </strong>the visual politics of play: on signifying practices of digital games. panel proposal (2p)</p> <p><strong>name name , name name. </strong>pain, art , communication. panel proposal (2p)</p> <p><strong>name name, name name , name name. </strong>waves of technology: hidden ideologies of cognitive neuroscience , future production of iconic. panel proposal (2p)</p>

the parsing works until last paragraph:

<p><strong>name name, name name , name name. </strong>waves of technology: hidden ideologies of cognitive neuroscience , future production of iconic. panel proposal (2p)</p>

what find in last slot of array this:

[u'name name, name name\xa0and name name.\xa0', u'waves\n of technology: hidden ideologies of cognitive neuroscience , \nfuture production of iconic.\xa0panel proposal (2p)', u'waves\n of technology: hidden ideologies of cognitive neuroscience , \nfuture production of iconic.\xa0panel proposal (2p)']

there 2 newlines (\n) appear in weird places (after waves , before future). appear in same position, not randomly. thought due length of paragraph, there longer paragraphs no \n appears.

i tried remove them with:

sprt[i][2] = re.sub("\n", "", sprt[i][1], re.u, re.s)

but didn't work.

are newlines there because made mistake somewhere? there way remove them?

i suspect newline appears in source html file. tried reproduce error using paragraphs , didn't \n until inserted new line in source file. explain why doesn't happen other longer paragraphs: don't have actual newline in html source file.

having said that, if add re.sub line newline character removed. (i in sprt[i][2] though, not sprt[i][1] of course - possible looking in wrong place there?)


Comments

Post a Comment

Popular posts from this blog

javascript - Bootstrap Popover: iOS Safari strange behaviour -

i looking through twitter bootstrap documentation, , i'm interested in implementing dismissible popover when button clicked. here example bootstrap's docs: http://getbootstrap.com/javascript/#dismiss-on-next-click when press "dismissible popover" button os x safari browser works fine. when access same link above ios safari, button not work. is, popover not dismissed on click. why button not working ios safari? the example linked using data-trigger="focus" attribute. try changing data-trigger="click" . <a tabindex="0" class="btn btn-lg btn-danger" role="button" data-toggle="popover" data-trigger="click" title="dismissible popover" data-content="and here's amazing content. it's engaging. right?">dismissible popover</a>
Read more

Magento/PHP - Get phones on all members in a customer group -

i working on custom magento extension. working around adminhtml part , i've created custom form there. here form code: <?php class vivasindustries_smsnotification_block_adminhtml_sms_sendmass_edit_form extends mage_adminhtml_block_widget_form { public function _preparelayout() { $extensionpath = mage::getmoduledir('js', 'vivasindustries_smsnotification'); $head = $this->getlayout()->getblock('head'); $head->addjs('jquery.js'); $head->addjs('vivas.js'); return parent::_preparelayout(); } protected function _prepareform() { $form = new varien_data_form(array( 'id' => 'edit_form', 'action' => $this->geturl('*/*/save', array('id' => $this->getrequest()->get...
Read more

session - Logging Out Using PHP -

i have seen other programs regarding can't seem make work. doing wrong? want achieve after user logged out, going previous pages become restricted. i have 3 php files: 1. sample.php - main page username , password filled in 2. successful.php - user's page after logging in 3. logout.php - destorys session hinder user access pages after logging out sample.php </html> <form action="" method="post"> <table> <tr> <td>username:</td> <td>password:</td> </tr> <tr> <td><input type="text" name="username"></td> <td><input type="password" name="password"></td> </tr> <tr> <td><input type="submit" value="submit" name="submit"></td> </tr> </table> </form> </html> <?php $servername = "localhost"; $username = ...
Read more