using unix/linux in puTTy

Consider the enrollment file IFT383_To_IA_FULL_20180108.csv. It was generated by combining multiple files created by individual students.

First thing first, well assume that not all the individual files have the same format.

Within text files, Unix marks the end of each line with a ^J (newline) character. Microsoft Windows, however, does it differently. Windows marks the end of each line with a ^M^J (in Unix terms, that would be return+linefeed). And the Macintosh uses a return (^M).

Thus, when you copy text files from Unix to Windows, each ^J must be changed to ^M^J. Conversely, when you files from Windows to Unix, each ^M^J must be changed to ^J.

Utility programs such as dos2unix and unix2dos are available to do these conversions. However, they may not be available on your remote system, and without root privileges, you might not be able to install them. So you must create your own alternate commands:

How do you convert a text file in Macintosh format to Unix format using tr?

tr unixfile

What if you have a text in Windows format and you want to use the file with Unix (In other words, how do you change the ^M^J into ^J)? In this case, we want to make change in the original file itself.

At this point, your file should be in Unix format.

Lets perform some pre-processing tasks. Provide the commands to check each of the below requirements. Be sure to use only the filters discussed in class.

Check that there is no duplicated SSN. If there is/are, display how many times each is duplicated

Check that the area code is exactly 3 digits and the home phone number is exactly 7 digits.

There shall be no space in any header name, except for the header name of the first field (i.e. Effective Date). Any other space should be replaced with underscore. For example, DOB Month should be DOB_Month

Were interested in how many primaries and dependents are to be enrolled

Were interested in how many Add, Drop, and Change are to be processed

Double quotes in the file must be remove. The output must be the same file (i.e. filename must not change)

Suppose you dont know how many fields are in a line, but you wish to use cut to extract the last field.

Now use the above technique to count the total number of Family primary plans

By default, sed performs its operations on every line in the data stream. To change this, you can preface your command with an address. This tells sed to operate only on the lines with that address.

For the records with Family Plan_Type, make all their status to Change. Save the resulting file as IFT394_Fall2017_To_InfoArmor_CAD_20170905_.csv. No piping allowed here.

Extract all the fields of the above file except the last one and display on the screen

You will now write a small program consisting of many sed commands.

Suppose you have a file named calendar in which you keep your schedule. Within the file, you have various abbreviations you would like to expand. In particular, you want to change mon to Monday.

Write the sed command to do so:

However, you also want to change tue to Tuesday. This requires two separate sed commands, both of which must be preceded by the e option.

Write the single-line sed command to perform the two tasks

Now you can see the pattern. Youre going to need seven separate sed commands, one for each day of the week. This would require a very long sed command line.

The best way to enter a very long command is to break it onto multiple lines. All you have to do is type a \ (backslash) before you press the key. The backslash quotes the newline, which allows you to break the command onto more than one line.

Using the \ to quote the newline, Write the long sed command that changes all the abbreviations for all seven days of the week in the file calendar and save it in a file called instructions.sed. Your program should show each sed command on its own line.