Art of BI: Removing ^M From Linux Files

By | In Art of BI, Big Data, Business Intelligence | January 20th, 2012

I was at a project recently where someone had updated several Linux config/text files using a Windows machine.
They FTP‘d the files to their desktop, used notepad or something similar to make the edits, and then FTP’d the file back to the Linux server thinking that all would work swimmingly. Unfortunately they overwrote the original files when they uploaded the files back to the Linux server.

Tux, the Linux penguin
Image via Wikipedia

As you know when this happens, Linux can recognize these foreign line breaks and places a nasty ^M at the end of each break as it interprets it as an actual character. This renders most config files, especially a XML based config file, useless for parsing by the program trying to read the file.

The solution is regular expression based and looks kinda funky but it works.

Basically open the file in question using the VI editor. Hit the colon command to start the menu input and enter the expression below against the file.

:%s/[ctrlkey+v and ctrl-key+M]//g

which will give you



Enhanced by Zemanta
Contact Us
Christian Screen
Christian is an innovator in analytics and data warehousing design, best practices, and delivery. With more than fifteenyears of decision support and data warehousing with key experiences at Office Depot HQ, Sierra-Cedar, and Capgemini, he oversees the Oracle Analytics Practice which includes the technical development and delivery of Oracle BI collaboration software, data warehouse solutions, Oracle BI/EPM projects, and packaged analytics solutions at Datavail.

Leave a Reply

Your email address will not be published.
Required fields are marked (*).

3 thoughts on “Art of BI: Removing ^M From Linux Files”
  1. Also use the following 2 commands : more and mv
    1>more old_file_name > temp_file_name
    Eg : more > a — where a is dynamic tmp file name.

    2>mv temp_file_name file_name
    Eg : mv a –reconvert a to original file