{"id":49985,"date":"2016-08-02T06:00:30","date_gmt":"2016-08-02T06:00:30","guid":{"rendered":"https:\/\/wordpress.org\/plugins-wp\/unfc-normalize\/"},"modified":"2017-01-19T10:31:16","modified_gmt":"2017-01-19T10:31:16","slug":"unfc-normalize","status":"publish","type":"plugin","link":"https:\/\/pcd.wordpress.org\/plugins\/unfc-normalize\/","author":13852463,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_crdt_document":"","version":"1.0.6","stable_tag":"1.0.6","tested":"4.7.33","requires":"3.9.13","requires_php":"","requires_plugins":"","header_name":"UNFC No\u0308rmalize","header_author":"gitlost","header_description":"","assets_banners_color":"b0b011","last_updated":"2017-01-19 10:31:16","external_support_url":"","external_repository_url":"","donate_link":"","header_plugin_uri":"https:\/\/github.com\/gitlost\/unfc-normalize","header_author_uri":"https:\/\/profiles.wordpress.org\/gitlost","rating":5,"author_block_rating":0,"active_installs":100,"downloads":2629,"num_ratings":2,"support_threads":0,"support_threads_resolved":0,"author_block_count":0,"sections":["description","installation","faq","changelog"],"tags":[],"upgrade_notice":{"1.0.6":"

Less noisy when WP_DEBUG set.<\/p>","1.0.5":"

Fixes fatal error on check slugs listing in DB check tool when Intl extension not available.<\/p>","1.0.4":"

Now adds _wp_old_slug on normalizing slugs so old links will work.<\/p>","1.0.3":"

Improved PHP 5 performance on isNormalized() check.<\/p>"},"ratings":{"1":0,"2":0,"3":0,"4":0,"5":"2"},"assets_icons":{"icon-128x128.png":{"filename":"icon-128x128.png","revision":"1465738","resolution":"128x128","location":"assets"},"icon-256x256.png":{"filename":"icon-256x256.png","revision":"1465738","resolution":"256x256","location":"assets"}},"assets_banners":{"banner-772x250.png":{"filename":"banner-772x250.png","revision":"1465738","resolution":"772x250","location":"assets"}},"assets_blueprints":{},"all_blocks":[],"tagged_versions":["1.0.1","1.0.2","1.0.3","1.0.4","1.0.5","1.0.6"],"block_files":[],"assets_screenshots":{"screenshot-1.png":{"filename":"screenshot-1.png","revision":"1542706","resolution":"1","location":"assets"},"screenshot-2.png":{"filename":"screenshot-2.png","revision":"1542706","resolution":"2","location":"assets"},"screenshot-3.png":{"filename":"screenshot-3.png","revision":"1542706","resolution":"3","location":"assets"},"screenshot-4.png":{"filename":"screenshot-4.png","revision":"1542706","resolution":"4","location":"assets"}},"screenshots":{"1":"Before and after: pasting non-normalized text into a post's title and content.","2":"Before: uploading a file on macOS with Firefox and then viewing with Safari (broken in Safari).","3":"After: uploading a file on macOS with Firefox and then viewing with Safari (whee it works).","4":"Front page of the database scan administration tool."},"jetpack_post_was_ever_published":false},"plugin_section":[],"plugin_tags":[46331,46330,30863,9654,46332],"plugin_category":[],"plugin_contributors":[126882,84669],"plugin_business_model":[],"class_list":["post-49985","plugin","type-plugin","status-publish","hentry","plugin_tags-form-c","plugin_tags-normalization","plugin_tags-normalize","plugin_tags-unicode","plugin_tags-unicode-normalization-form-c","plugin_contributors-gitlost","plugin_contributors-zodiac1978","plugin_committers-gitlost"],"banners":{"banner":"https:\/\/ps.w.org\/unfc-normalize\/assets\/banner-772x250.png?rev=1465738","banner_2x":false,"banner_rtl":false,"banner_2x_rtl":false},"icons":{"svg":false,"icon":"https:\/\/ps.w.org\/unfc-normalize\/assets\/icon-128x128.png?rev=1465738","icon_2x":"https:\/\/ps.w.org\/unfc-normalize\/assets\/icon-256x256.png?rev=1465738","generated":false},"screenshots":[{"src":"https:\/\/ps.w.org\/unfc-normalize\/assets\/screenshot-1.png?rev=1542706","caption":"Before and after: pasting non-normalized text into a post's title and content."},{"src":"https:\/\/ps.w.org\/unfc-normalize\/assets\/screenshot-2.png?rev=1542706","caption":"Before: uploading a file on macOS with Firefox and then viewing with Safari (broken in Safari)."},{"src":"https:\/\/ps.w.org\/unfc-normalize\/assets\/screenshot-3.png?rev=1542706","caption":"After: uploading a file on macOS with Firefox and then viewing with Safari (whee it works)."},{"src":"https:\/\/ps.w.org\/unfc-normalize\/assets\/screenshot-4.png?rev=1542706","caption":"Front page of the database scan administration tool."}],"raw_content":"\n

This is a souped-up version of the Normalizer plugin<\/a> by Torsten Landsiedel<\/a>.<\/p>\n\n

It adds WP filters to normalize UTF-8 data coming into the system to the de facto web standard Normalization Form C<\/a>. The Unicode Consortium report is at Unicode Normalization Forms<\/a>.<\/p>\n\n

Polyfills<\/h4>\n\n

For best performance install<\/a> (if possible) the PHP Internationalization extension Intl<\/code><\/a>, which includes the PHP class Normalizer<\/code>.<\/p>\n\n

However the plugin works without the PHP Intl<\/code> extension being installed, as it uses (a modified version of) the Symfony Normalizer<\/code> polyfill<\/a>.<\/p>\n\n

Also text pasted into (most) inputs is normalized immediately using the JavaScript normalize()<\/code> method<\/a>. For browsers without normalization support, the unorm polyfill<\/a> is used.<\/p>\n\n

Background<\/h4>\n\n

For further info, see the WP Trac ticket #30130 Normalize characters with combining marks to precomposed characters<\/a> and this Make WP Core comment<\/a>.<\/p>\n\n

The plugin also works around the Safari-specific issues raised by the WP Trac ticket #22363 Accents in attachment filenames should be sanitized<\/a>.<\/p>\n\n

Scanner<\/h4>\n\n

For existing data, the plugin includes an administration tool to scan and normalize the database. Important:<\/strong> before using this tool to normalize, please backup your database<\/a>. This is especially important if your database contains non-normalized serialized data, as this plugin uses the same suck-and-see technique as interconnect\/it's Database Search and Replace Script in PHP<\/a> to deal with serialized data, and is fallible.<\/p>\n\n

And<\/h4>\n\n

A google-cheating schoolboy French translation is supplied.<\/p>\n\n

The plugin should run on PHP 5.2.17 to 7.1, and on WP 3.9.13 to 4.7.1.<\/p>\n\n

The project is on github<\/a>.<\/p>\n\n\n

Install in the standard way via the 'Plugins' menu in WordPress and then activate. No further action is needed.<\/p>\n\n\n

\n
How can I normalize extra stuff?<\/dt>\n

You can add normalization to anything that passes its content through a filter. The canonical way is to use the unfc_extra_filters<\/code> filter which returns an array of filter names - for instance, in your theme's functions.php<\/code> file, add:<\/p>\n\n

function mytheme_unfc_extra_filters( $extra_filters ) {\n    $extra_filters[] = 'myfilter';\n    return $extra_filters;\n}\nadd_filter( 'unfc_extra_filters', 'mytheme_unfc_extra_filters' );\n<\/code><\/pre>\n\n
Note that the unfc_extra_filters<\/code> filter is only called in the administration backend. You can also add a filter directly, to be called in the frontend or backend, by referencing the global PHP variable unfc_normalize<\/code>, but you should ensure that the Normalizer<\/code> polyfill is loaded if you don't have the PHP Intl<\/code> extension installed:<\/p>\n\n
global $unfc_normalize;\nif ( $unfc_normalize ) {\n    if ( ! function_exists( 'normalizer_is_normalized' ) ) { \/\/ If the \"Intl\" extension is not installed...\n        $unfc_normalize->load_unfc_normalizer_class(); \/\/ ...load the polyfill.\n    }\n    add_filter( 'myfilter', array( $unfc_normalize, 'normalize' ), 6 \/* Or whatever priority you choose *\/ );\n}\n<\/code><\/pre><\/dd>\n
Does it have a tool to normalize the filenames of files uploaded prior to its enabling?<\/dt>\n
No, it doesn't. Alas, such files would have to re-uploaded. However if (enough - or more like any) interest were expressed...<\/p><\/dd>\n\n<\/dl>\n\n\n
1.0.6<\/h4>\n\n
    \n
  • Require explicit define of UNFC_DEBUG. Move load of debug file to 'init'.<\/li>\n
  • Safari submit workaround.<\/li>\n
  • WP 4.7.1 compatible.<\/li>\n<\/ul>\n\n
    1.0.5<\/h4>\n\n
      \n
    • Do load of normalizer in slugs list constructor not items list. Fixes slugs listing crash when Intl extension not installed.<\/li>\n
    • Ignore case when checking charset (UNFC_Normalize::is_blog_utf8()).<\/li>\n
    • WP coding standards (yoda style in unfc_utf8_4range()).<\/li>\n
    • Add screenshots.<\/li>\n
    • Some extra unittests.<\/li>\n
    • WP 4.7.0 compatible.<\/li>\n<\/ul>\n\n
      1.0.4<\/h4>\n\n
        \n
      • Add _wp_old_slug on normalizing slugs.<\/li>\n
      • Escape title in screen reader label in db check.<\/li>\n<\/ul>\n\n
        1.0.3<\/h4>\n\n
          \n
        • For PHP 5 performance do preliminary preg_match on isNormalized. <\/li>\n
        • Improve comments in Normalizer.php, tabs -> 4 spaces, UNFC_REGEX_IS_INVALID_UTF8_XXX invert & rename.<\/li>\n
        • Fix untested admin notice and adjust tests. Fix some test bleed ($wp_scripts).<\/li>\n<\/ul>\n\n
          1.0.2<\/h4>\n\n
            \n
          • Move all .php files bar main to includes subdir. Remove unused ajax.<\/li>\n
          • Fix single-byte trie. Move regex alts to tools\/functions.php.<\/li>\n
          • Fix untested admin notice. Adjust tests for untested admin_notice change. Fix some test bleed ($wp_scripts).<\/li>\n
          • WP 4.6 compatible.<\/li>\n<\/ul>\n\n
            1.0.1<\/h4>\n\n
              \n
            • First release for wordpress.org repository.<\/li>\n
            • Add assets.<\/li>\n
            • Include \"class-unfc-list_table.php\" in .pot file (for forward\/backward compat).<\/li>\n
            • Remove unused variable in UNFC_DB_Check_Slugs_List_Table.<\/li>\n<\/ul>\n\n
              1.0.0<\/h4>\n\n
                \n
              • Initial release.<\/li>\n<\/ul>\n\n
                0.9.1<\/h4>\n\n
                  \n
                • Fix bad serialized data corruption on db check.<\/li>\n
                • Fix text domain tag.<\/li>\n<\/ul>\n\n
                  0.9.0<\/h4>\n\n
                    \n
                  • Initial version after renaming from tl-normalize.<\/li>\n<\/ul>","raw_excerpt":"Normalizes UTF-8 input to Normalization Form C.","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin\/49985","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin"}],"about":[{"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/types\/plugin"}],"replies":[{"embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/comments?post=49985"}],"author":[{"embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wporg\/v1\/users\/gitlost"}],"wp:attachment":[{"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/media?parent=49985"}],"wp:term":[{"taxonomy":"plugin_section","embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_section?post=49985"},{"taxonomy":"plugin_tags","embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_tags?post=49985"},{"taxonomy":"plugin_category","embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_category?post=49985"},{"taxonomy":"plugin_contributors","embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_contributors?post=49985"},{"taxonomy":"plugin_business_model","embeddable":true,"href":"https:\/\/pcd.wordpress.org\/plugins\/wp-json\/wp\/v2\/plugin_business_model?post=49985"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}