python - Remove byte order mark from objects in a list -


i using python (3.4, on windows 7) download set of text files, , when read (and write, after modifications) these files appear have few byte order marks (bom) among values retained, utf-8 bom. use each text file list (or string) , cannot seem remove these bom. ask whether possible remove bom?

for more context, text files downloaded public ftp source users upload own documents, , original encoding highly variable , unknown me. allow download run without error, specified encoding utf-8 (using latin-1 give errors). it's not mystery me have bom, , don't think up-front encoding/decoding solution answer me (convert utf-8 bom utf-8 no bom in python) - appears make frequency of other bom increase.

when modify files after download, use following syntax:

with open(t, "w", encoding='utf-8') outfile:     open(f, "r", encoding='utf-8') infile:         text = infile.read         #arguments make modifications follow

later on, after "outfiles" read in list see words have utf-8 bom, \ufeff. try remove bom using following list comprehension:

g = list_outfile    #outfiles stored list g = [i.replace(r'\ufeff','') in g]

while argument run, unfortunately bom remain when, example, print list (i believe have similar issue if tried remove bom strings , not lists: how remove special character?). if put normal word (non-bom) in list comprehension, word replaced.

i understand if print list object object bom not appear (special national characters won't .split() in python). , bom not in raw text files. worry bom remain when running later arguments text analysis , object appears in list \ufeffword rather word analyzed \ufeffword.

again, possible remove bom after fact?


Comments

Post a Comment

Popular posts from this blog

javascript - Bootstrap Popover: iOS Safari strange behaviour -

i looking through twitter bootstrap documentation, , i'm interested in implementing dismissible popover when button clicked. here example bootstrap's docs: http://getbootstrap.com/javascript/#dismiss-on-next-click when press "dismissible popover" button os x safari browser works fine. when access same link above ios safari, button not work. is, popover not dismissed on click. why button not working ios safari? the example linked using data-trigger="focus" attribute. try changing data-trigger="click" . <a tabindex="0" class="btn btn-lg btn-danger" role="button" data-toggle="popover" data-trigger="click" title="dismissible popover" data-content="and here's amazing content. it's engaging. right?">dismissible popover</a>
Read more

Magento/PHP - Get phones on all members in a customer group -

i working on custom magento extension. working around adminhtml part , i've created custom form there. here form code: <?php class vivasindustries_smsnotification_block_adminhtml_sms_sendmass_edit_form extends mage_adminhtml_block_widget_form { public function _preparelayout() { $extensionpath = mage::getmoduledir('js', 'vivasindustries_smsnotification'); $head = $this->getlayout()->getblock('head'); $head->addjs('jquery.js'); $head->addjs('vivas.js'); return parent::_preparelayout(); } protected function _prepareform() { $form = new varien_data_form(array( 'id' => 'edit_form', 'action' => $this->geturl('*/*/save', array('id' => $this->getrequest()->get...
Read more

session - Logging Out Using PHP -

i have seen other programs regarding can't seem make work. doing wrong? want achieve after user logged out, going previous pages become restricted. i have 3 php files: 1. sample.php - main page username , password filled in 2. successful.php - user's page after logging in 3. logout.php - destorys session hinder user access pages after logging out sample.php </html> <form action="" method="post"> <table> <tr> <td>username:</td> <td>password:</td> </tr> <tr> <td><input type="text" name="username"></td> <td><input type="password" name="password"></td> </tr> <tr> <td><input type="submit" value="submit" name="submit"></td> </tr> </table> </form> </html> <?php $servername = "localhost"; $username = ...
Read more