{"id":3330,"date":"2022-09-12T15:07:29","date_gmt":"2022-09-12T19:07:29","guid":{"rendered":"https:\/\/www.alpharithms.com\/?p=3330"},"modified":"2022-09-12T15:07:41","modified_gmt":"2022-09-12T19:07:41","slug":"input-field-separators-ifs","status":"publish","type":"post","link":"https:\/\/www.alpharithms.com\/input-field-separators-ifs-290715\/","title":{"rendered":"Input Field Separators (IFS): Turning Strings into Words"},"content":{"rendered":"

Internal Field Separators (IFS) are characters or sequences or characters by which a larger sequence or pattern of characters can be tokenized. This process of tokenization is commonly done via the space, tab, and newline characters to produce distinct boundaries between unique words within a body of text.<\/p>\n

For example, standard IFS-mediated tokenization of the character string (showing space characters explicitly) the\\slazy\\sbrown\\sfox<\/code> will result in the collection of individual words (a.k.a.\u00a0tokens<\/em>) [the<\/code>, lazy<\/code>, brown<\/code>, fox<\/code>] where the \\s<\/code> characters indicate a single space.<\/p>\n

IFS in Bash & Other Shells<\/h2>\n

Command line interpreters or, as they are commonly referred to; shells,<\/em>\u00a0 provide users with the means of interacting with a system. In the specific case of bash, the string of characters IFS<\/code> is a system-reserved keyword (a.k.a token) that represents the Internal Field Separator used by the bash program. The IFS is a variable in that its value can be changed. By default, it is the three-character sequence of tab, space, newline (\\t, \\s, \\n).<\/p>\n

To see an example of IFS in action, consider the following bash script:<\/p>\n

#!\/bin\/bash\r\n\r\n# define a string variable\r\nstring=\"one:two three four-five\"\r\n\r\n# iterate over each \"word\" as \r\n# determined by use of IFS\r\nfor word in $string; do\r\n\r\n  # print each token\r\n  echo \"Word: $word\"\r\ndone<\/pre>\n
Here we iterate over every “word” within the string one:two three four-five<\/code> of which are determined to be separate, distinct entities by the value of IFS<\/code>. Since the \\s<\/code> single space character is in the default IFS value, this string is split on the single space resulting in the following printout:<\/p>\n
Word: one:two\r\nWord: three    \r\nWord: four-five\r\n<\/pre>\n
Here we see the entire string having been tokenized (a.k.a.\u00a0split<\/em>) at each occurrence of a single space, resulting in three unique sequences of characters. This use is the common case and illustrative of why the default IFS value was chosen as such. However, as mentioned, the IFS variable can reference a custom value. Consider the same script ran above with a new addition of a custom value for the IFS variable:<\/p>\n
#!\/bin\/bash\r\n\r\n# define a string variable\r\nstring=\"one:two three four-five\"\r\n\r\n# define custom IFS value\r\nIFS=\":-\"\r\n\r\n# iterate over word boundaries\r\nfor word in $string; do\r\n  echo \"Word: $word\"\r\ndone<\/pre>\n
Word boundaries are now defined to be either :<\/code> or -<\/code> such that single spaces are considered interword characters. The output of this script is as follows:<\/p>\n
Word: one\r\nWord: two three four\r\nWord: five\r\n<\/pre>\n
Here we see the entire two three four<\/code> sequence is being considered a single “word” since the IFS value no longer contains the single space. Instead, the string gets split on the :<\/code> and -<\/code> characters.<\/p>\n
Final Thoughts<\/h2>\n
Internal Field Separators are used across many programming languages and development environments. In some cases, the term can be seen written as Input Field Separator which, arguably, is a rough equivalent albeit slightly more specific. There are a number of exploits related to the IFS that adversaries leverage. Most modern shells have addressed these issues.<\/p>\n","protected":false},"excerpt":{"rendered":"
Internal Field Separators (IFS) are characters or sequences or characters by which a larger sequence or pattern of characters can be tokenized. This process of tokenization is commonly done via the space, tab, and newline characters to produce distinct boundaries between unique words within a body of text. For example, standard IFS-mediated tokenization of the […]<\/p>\n","protected":false},"author":2,"featured_media":3333,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[164,629,260],"tags":[736,12,737],"acf":[],"_links":{"self":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/3330"}],"collection":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/comments?post=3330"}],"version-history":[{"count":3,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/3330\/revisions"}],"predecessor-version":[{"id":3334,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/posts\/3330\/revisions\/3334"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media\/3333"}],"wp:attachment":[{"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/media?parent=3330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/categories?post=3330"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.alpharithms.com\/wp-json\/wp\/v2\/tags?post=3330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}