Upgrading to Joomla4

After updating the PHP to 8.1 or PHP 8.2 it is time to upgrade as well the Joomla version from the latest stable version Joomla 3.10.11 Stable to Joomla 4. This process is usually pain but you need to watch out for incompatible plugins and modules. During the PHP update, we had the Back End still accessible, however, during upgrading to Joomla 4 an incompatible plugin or module or component can cause the Back End to be as dead as the Front End. This can cause a lot of issues since uninstalling the plugin after can be a pain. We will address this issue and try to fix the extension until the moment we can at least uninstall it. However, I recommend uninstalling anything that Joomla 4 upgrade script will point as a risk, you will save yourself a head ache.

We are however living on the edge and thus we will keep everything there to see with what we will end up. To make things a bit safer for us I will recommend using Akeeba Backup Core and backing up the site. This will allow us to revert the changes by using the backup file. To make it even better for us, I would suggest installing XAMPP locally and testing the set locally using the backup before upgrading the live site. After this initial set-up, we can proceed with updating the local site, or if you are brave enough, go for the live site straight.

First, you need to update to the latest stable version which is Joomla 3.10.11 Stable. After updating the site to the latest stable release of Joomla 3.x proceed to Joomla Update located in Components-Joomla! Update, and in options change the update channel to Joomla Next, save, and close. Straight after saving these settings, a pre-check will be automatically run to ensure your site is ready to be upgraded. You will end up with a list of plugins, modules, and components which are necessary to be updated or removed. Let's have a look at them.

A lot of these plugins, modules, and components can be installed again like the ARK editor and hence it is not necessary to keep it there since we can install it again. At this point I will uninstall some of the extensions that can be safely installed again and will not make any changes to the live site. After unistalling what I do not need I was left with: Kick GDPR Cookie-Plugin, Show attachments in editor, and Yjsg Framework. Now the last one is the most crucial for now since it is the framework that runs the live site right now. The attachments are necessery since they are the files to the content, right? However, the cookie plugin is something we could keep away but keeping it for now. We will get on these extensions "Potential Upgrade Issue" but since we are brave, we will ignore it. Go to the next tab called "Live Update" and check the "Do you wish to ignore the warnings about potentially incompatible plugins and to proceed with the upgrade?" checkbox, since we are brave.

Sit in your chair and keep the update process working on upgrading your site. At this moment, waiting and praying is the only thing you can do. We are upgrading to the latest version of Joomla 4.2.9. Well, as you can see below my site went from nice to broken in a few seconds. Both the Back End and the Front End are broken and this means either revert the backup file and start over, uninstalling the incompatible stuff, or try to fix them and then uninstall them since they will be doing issues later on - or fix them but I am not sure if it is worth the time especially when those extensions are not developed further.

As before, let's turn DEBUG for Joomla to see where the problems lie. Since the Back End is broken we will log via FTP to the root of the site or just navigate locally to the location of the site and open the configuration.php and edit the debug line, as below. We need to see where the problem and errors are and this is the only option to see it.

configuration.php
#Find line
$debug = '0'; #and change '0' to '1'

Let's start with the Back End since working Back End could save us by uninstalling the incompatible extensions. With DEBUG on we can see again an issue in the YJSG framework, no surprise, right? The problem is now different. Before we had PHP issues but now we have Joomla error which indicates that the right documentation to look in to is for Joomla 4 and not PHP.

0 Call to undefined method Joomla\CMS\Application\AdministratorApplication::isAdmin()

Let's start with the Back End since working Back End could save us by uninstalling the incompatible extensions. With DEBUG on we can see again an issue in the YJSG framework, no surprise, right? The problem is now different. Before we had PHP issues but now we have a Joomla error which indicates that the right documentation to look into is for Joomla 4 and not PHP. The error we see at first is referring to the declaration of the user level. After some reading you will find that ::isAdmin() was changed to isClient("administrator") early in Joomla 3.x but was completely removed from Joomla 4. A similar error is referring to isSite() since it was changed to isClient('site'). You can do a mass replacement in the whole file for those errors.

if ($this->app->isAdmin()) {
#change to
if ($this->app->isClient("administrator")) 
#
if ($this->app->isSite() && $this->run_plg == 1) {
#change to
if ($this->app->isClient('site') && $this->run_plg == 1) {{

The next error we are getting seems to be more serious. It refers to a missing databse table, what, how, why???? Specifically, the Back End is referring to a missing 500 Table '3d2.ar_workflow_associations' that doesn't exist. Now, this is interesting since we would assume that only errors related to extensions will be present but right now an error for missing tables is bothering us. The only thing we can do now is to log in phpMyAdmin and try to verify if the _workflow_associations table exists or not.

500 Table '3d2.ar_workflow_associations' doesn't exist

In my case, the table doesn't exist so let's create it. Please, make sure you will back up as you are proceeding, otherwise, you can end up with a broken site with no turnback. If you are sure you want to proceed, let's create the tables that are missing. The first one is already mentioned 3d2.ar_workflow_associations. The 3d2 is the database, and ar_ is the prefix. This can be different in your case, change this to your database setup. The following SQL queries will run the creation of the tables as listed below:

• 3d2.ar_workflow_associations
• 3d2.ar_workflow_stages
• 3d2.ar_workflows

CREATE TABLE `ar_workflow_associations` (
  `item_id` int(11) NOT NULL DEFAULT 0 COMMENT 'Extension table id value',
  `stage_id` int(11) NOT NULL COMMENT 'Foreign Key to #__workflow_stages.id',
  `extension` varchar(50) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

CREATE TABLE `ar_workflow_stages` (
  `id` int(11) NOT NULL,
  `asset_id` int(11) DEFAULT 0,
  `ordering` int(11) NOT NULL DEFAULT 0,
  `workflow_id` int(11) NOT NULL,
  `published` tinyint(4) NOT NULL DEFAULT 0,
  `title` varchar(255) NOT NULL,
  `description` text NOT NULL,
  `default` tinyint(4) NOT NULL DEFAULT 0,
  `checked_out_time` datetime DEFAULT NULL,
  `checked_out` int(10) UNSIGNED DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

CREATE TABLE `ar_workflows` (
  `id` int(11) NOT NULL,
  `asset_id` int(11) DEFAULT 0,
  `published` tinyint(4) NOT NULL DEFAULT 0,
  `title` varchar(255) NOT NULL,
  `description` text NOT NULL,
  `extension` varchar(50) NOT NULL,
  `default` tinyint(4) NOT NULL DEFAULT 0,
  `ordering` int(11) NOT NULL DEFAULT 0,
  `created` datetime NOT NULL,
  `created_by` int(11) NOT NULL DEFAULT 0,
  `modified` datetime NOT NULL,
  `modified_by` int(11) NOT NULL DEFAULT 0,
  `checked_out_time` datetime DEFAULT NULL,
  `checked_out` int(10) UNSIGNED DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

The next error is again database related 500 Unknown column 'fp.featured_up' in 'field list'. This means that we are missing some columns in ar_content_frontpage table. Well, let's add them since it refers to featured_up we can guess that featured_down will be missing as well. The queries below will add them one by one. After fixing this, I am back to "normal" errors like 0 Call to undefined method Joomla\CMS\Application\AdministratorApplication::isSite() and since we know how to fix them, let's do it. The error is again referring to  JROOT\plugins\system\yjsg\yjsg.php:984 and thus YJSG plugin.

ALTER TABLE `ar_content_frontpage` ADD `featured_up` DATETIME;
ALTER TABLE `ar_content_frontpage` ADD `featured_down` DATETIME;

The next error I got is not from the YJSG plugin but from the Attachment extension. The error simply states 0 Class "JRequest" not found. Looking at the line of code and reading something about JRequest and Joomla 4 you will find out that  JRequest is deprecated. You can see that you need to fetch a variable and then replace the code with something like below.

$task = JRequest::getCmd('task');
#change to
$jinput = JFactory::getApplication()->input;
$task = $jinput->get('task');

Now, let's get a bit to the Front End since we do have still errors here and there. The error that is now shining on us stating Unknown column 'm.publish_up' in 'where clause' is again related to a missing column in a table. This means that are update went so badly that not all SQL queries went as they were intended. Caused again by extensions. This missing column refers to the ar_menu table where two columns are missing, see below. You can add them using the SQL queries listed below one by one. After adding these missing columns, voila, at least some error on the Front Page related to some code issue with Joomla 4 rather than an issue with the database.

ALTER TABLE `ar_menu` ADD `publish_up` DATETIME;
ALTER TABLE `ar_menu` ADD `publish_down` DATETIME;

Since we have only a code error on the Front End, let's come back to the Back End. It seems that there is still some error after choosing a specific function within the Back End related to the "attachment" extension but what is bothering me more is the missing template of the Back End. It looks like it is without any CSS or anything. This means that a lot was missed during the upgrade. At this moment I would most likely start over and think twice if to keep the unsupported extensions and just uninstall them, however, we are here to solve the issue or at least to try to solve it since it can happen. If we look at how the Back End is looking it may point us to a missing template, and thus let's get back to phpMyAdmin and look at ar_template_styles table. Now, we can see that we are missing the new templates that come with Joomla 4 and are stuck with the old ones. What we would expect is the atum and cassiopeia template. Physically I see them in the appropriate folders but no entry what so over.

Now we know the templates are missing and thus let's bring them back. The SQL query below will add new entries within the ar_template_styles table with ID 10 and ID 11. Make sure these IDs are not occupied. Keep everything as it is for now. Looking at the tables, yes, I have a fresh copy of Joomla 4 next to this instance to see what is missing, I see that we miss another table in the database structure called ar_template_overrides. Let's add it as well with the SQL query below. To load the Back End template I had to change the client_id and home values from 1 to 0 within phpMyAdmin - ar_template_styles by the isis template and add these values to atum.

INSERT INTO `ar_template_styles` (`id`, `template`, `client_id`, `home`, `title`, `inheritable`, `parent`, `params`) VALUES
(10, 'atum', 1, '1', 'Atum - Default', 1, '', '{\"hue\":\"hsl(214, 63%, 20%)\",\"bg-light\":\"#f0f4fb\",\"text-dark\":\"#495057\",\"text-light\":\"#ffffff\",\"link-color\":\"#2a69b8\",\"special-color\":\"#001b4c\",\"monochrome\":\"0\",\"loginLogo\":\"\",\"loginLogoAlt\":\"\",\"logoBrandLarge\":\"\",\"logoBrandLargeAlt\":\"\",\"logoBrandSmall\":\"\",\"logoBrandSmallAlt\":\"\"}'),
(11, 'cassiopeia', 0, '1', 'Cassiopeia - Default', 1, '', '{\"brand\":\"1\",\"logoFile\":\"\",\"siteTitle\":\"\",\"siteDescription\":\"\",\"useFontScheme\":\"0\",\"colorName\":\"colors_standard\",\"fluidContainer\":\"0\",\"stickyHeader\":0,\"backTop\":0}');

CREATE TABLE `ar_template_overrides` (
  `id` int(10) UNSIGNED NOT NULL,
  `template` varchar(50) NOT NULL DEFAULT '',
  `hash_id` varchar(255) NOT NULL DEFAULT '',
  `extension_id` int(11) DEFAULT 0,
  `state` tinyint(4) NOT NULL DEFAULT 0,
  `action` varchar(50) NOT NULL DEFAULT '',
  `client_id` tinyint(3) UNSIGNED NOT NULL DEFAULT 0,
  `created_date` datetime NOT NULL,
  `modified_date` datetime DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; 

ALTER TABLE `ar_template_overrides`
  ADD PRIMARY KEY (`id`),
  ADD KEY `idx_template` (`template`),
  ADD KEY `idx_extension_id` (`extension_id`);

ALTER TABLE `ar_template_overrides` MODIFY `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT;

The admin menu with CSS is back after our changes in the database and phpMyAdmin and this gives us hope to proceed with the fixing. The Menu is almost empty, with no modules whatsoever, and by trying to add a new one an error will appear referring to the extension attachments with an error message Call to undefined method Joomla\CMS\Factory::getURI(). The problem is now causing the extension since it contains a code Factory::getURI() that does not exist anymore in Joomla 4.

Call to undefined method Joomla\CMS\Factory::getURI()

From the debug window we know that the problematic file is located here:   JROOT\plugins\content\attachments\attachments.php:51, and thus we will try to fix this. This will actually solve the error and we can start to add modules within the administration panel. However, straight after I found another issue Class "JResponse" was not found in, and again, the attachment extension. Since "JResponse" was replaced by "JFactory" we need to alter this as well, as below. After this, I have got most of the Back End functions ready, as I see although I am still missing the template menu.

#At first you need to define something at the beginning of the document to
#load the appropriate functions
#Add after defined('_JEXEC') or die('Restricted access');
#
use Joomla\CMS\Uri\Uri;
#
#Then find the issue and replace
$uri= JFactory::getURI();
#change to
$$uri = Uri::getInstance();

$body = JResponse::getBody();
#change to
$body = JFactory::getApplication()->getBody();

JResponse::setBody($body);
#change to
JFactory::getApplication()->setBody($body);

It seems that even after all we had done there are still some errors and most likely within the database. Remember that you can access certain menu links using a link and you don't need a button for it, look below. Joomla 4 has built tools to check the database structure and as well the option to reinstall the core. Let's try both options to see if that will help us to resolve the problem. However, at first, let's uninstall the non-supported extensions to avoid more errors during the database check and update of the core. Use the link below to access the menu and uninstall what was unsupported. I have started with attachments which during the uninstall returned Call to the undefined method Joomla\Database\Mysqli\MysqliDriver::query(). Well, let's fix this to continue.

#Access the manage extension menu  
administrator/index.php?option=com_installer&view=manage

Call to undefined method Joomla\Database\Mysqli\MysqliDriver::query()

$db->query(); 
#change to
by $db->execute();

After uninstalling what was left we can jump to the database and Joomla core reinstallation menu. For that use the links below and run the appropriate functions. The database menu will show you some errors in the structure, let's fix them and hope for the best. After this run, the Joomla core update, and again, let's pray it will make some difference. After both, I have got still an issue in the database saying that I am missing a table. Well, let's add the table. Namely, the missing table was ar_finder_taxonomy. Adding a table is not a problem and we can use the queries below to do so. Adding the table helped but still not resolved everything. I had to alter a bit more other tables according to the Joomla 4 structure and as well add content to ar_workflow_stages, ar_workflow_transitions, ar_workflow_associations, and ar_workflows. I was missing the associations to the content whatsoever and it was a pain to add. However, in the end, the site was back online in the Back End and Front End.

CREATE TABLE `ar_finder_taxonomy` (
  `id` int(10) UNSIGNED NOT NULL,
  `parent_id` int(10) UNSIGNED NOT NULL DEFAULT 0,
  `lft` int(11) NOT NULL DEFAULT 0,
  `rgt` int(11) NOT NULL DEFAULT 0,
  `level` int(10) UNSIGNED NOT NULL DEFAULT 0,
  `path` varchar(400) NOT NULL DEFAULT '',
  `title` varchar(255) NOT NULL DEFAULT '',
  `alias` varchar(400) NOT NULL DEFAULT '',
  `state` tinyint(3) UNSIGNED NOT NULL DEFAULT 1,
  `access` tinyint(3) UNSIGNED NOT NULL DEFAULT 1,
  `language` char(7) NOT NULL DEFAULT ''
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;. 

INSERT INTO `ar_finder_taxonomy` (`id`, `parent_id`, `lft`, `rgt`, `level`, `path`, `title`, `alias`, `state`, `access`, `language`) VALUES
(1, 0, 0, 1, 0, '', 'ROOT', 'root', 1, 1, '*');

ALTER TABLE `ar_finder_taxonomy`
  ADD PRIMARY KEY (`id`),
  ADD KEY `idx_state` (`state`),
  ADD KEY `idx_access` (`access`),
  ADD KEY `idx_path` (`path`(100)),
  ADD KEY `idx_level` (`level`),
  ADD KEY `idx_left_right` (`lft`,`rgt`),
  ADD KEY `idx_alias` (`alias`(100)),
  ADD KEY `idx_language` (`language`),
  ADD KEY `idx_parent_published` (`parent_id`,`state`,`access`);

ALTER TABLE `ar_finder_taxonomy`
  MODIFY `id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=2;

Conclusion

The upgrading process to Joomla 4 can be easy or it can be a nightmare. During this process, I would always suggest backup the site before any changes and uninstalling everything which is not compatible with the new version or can cause a problem. Next to the above, I would run the update first locally and then on a live site, you can save yourself a lot of issues with the process.

However, we know that for any problem there is a solution, it can take some time to solve but it is certainly solvable, and therefore if you will end up with a broken one, take a deep breath and fix it one by one. Good luck with the upgrade and fixing and I hope the article was useful for you.