UnmailingDaily

Wednesday, January 31, 2018

Mail backup.

Code reviews.

Some thinking and mail exchanges on spectre / meltdown, notably with respect to migration and VM usage.

Installing Redmine

I started with these instructions, but they seem quite outdated. Summary of what I know needs to be done to install:

Let's start with the dependencies:

yum -y install zlib-devel curl-devel openssl-devel httpd-devel apr-devel apr-util-devel mysql-devel

Then since I'd like to use postgresql and not MySQL or MariaDB, let's try that first:

yum install postgresql postgresql-server

Open the firewall for Apache (port 80), otherwise access is denied by default:

firewall-config

Ruby is installed, and I initially thought that the built-in v2.0 was OK:

% ruby -v
ruby 2.0.0p648 (2015-12-16) [x86_64-linux]

This gives me a correct version of GEM:

% gem -v
2.0.14.1

But it turns out that Passenger install fails:

% gem install passenger
Fetching: rake-12.3.0.gem (100%)
Successfully installed rake-12.3.0
Fetching: rack-2.0.3.gem (100%)
ERROR:  Error installing passenger:
	rack requires Ruby version >= 2.2.2.

So it turns out the correct solution is to manually upgrade Ruby to the latest stable:

% wget https://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.0.tar.gz
% tar xvfz ruby-2.5.0.tar.gz
% cd ruby-2.5.0/
% ./configure
% make

Then you get:

% ruby -v
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]

And then installing Passenger works:

% gem install passenger
gem install passenger
Fetching: rack-2.0.3.gem (100%)
Successfully installed rack-2.0.3
Fetching: passenger-5.2.0.gem (100%)
Building native extensions. This could take a while...
Successfully installed passenger-5.2.0
Parsing documentation for rack-2.0.3
Installing ri documentation for rack-2.0.3
Parsing documentation for passenger-5.2.0
Installing ri documentation for passenger-5.2.0
Done installing documentation for rack, passenger after 53 seconds
2 gems installed

Then the rest of the installation works semi-OK:

% cd /var/www
% svn co http://svn.redmine.org/redmine/branches/3.4-stable redmine
% systemctl restart httpd

Installing PostgreSQL

Check that postgresql is running OK:

% psql
psql: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

So need to install and setup:

% sudo yum install postgresql-server
% posgresql-setup initdb
Initializing database ... OK
% systemctl start postgresql
% systemctl enable postgresql

For postgres, the super-user is user postgres:

% su - postgres
% psql
postgres=# alter role postgres with encrypted password 'stuff';
postgres=# create user redmine with encrypted password 'stuff2';

Creating the database with UTF-8 encoding fails:

postgres=# create database redmine with encoding 'UTF-8';
ERROR:  new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII)
HINT:  Use the same encoding as in the template database, or use template0 as template.

So you need:

postgres=# create database redmine with template = template0 encoding 'UTF-8';

I also found an alternative that, if I understand correctly, recreates template1 to default to allowing this kind of creation:

postgres=# update pg_database set datallowconn = TRUE where datname = 'template0';
UPDATE 1
postgres=# c template0
You are now connected to database "template0" as user "postgres".
template0=# update pg_database set datistemplate = FALSE where datname = 'template1';
UPDATE 1
template0=# drop database template1;
DROP DATABASE
template0=# create database template1 with template = template0 encoding = 'UTF8';
CREATE DATABASE
template0=# update pg_database set datistemplate = TRUE where datname = 'template1';
UPDATE 1
template0=# c template1
You are now connected to database "template1" as user "postgres".
template1=# update pg_database set datallowconn = FALSE where datname = 'template0';
UPDATE 1
template1=# create database redmine with encoding 'UTF-8';
CREATE DATABASE

After that, the original redmine database creation works.

It's probably a good idea to create the database so that owner is redmine, see below.

MariaDB alternative

The alternative for MySQL is:

% mysql -u root -p
Enter password:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)

Installing MariaDB solves it:

yum insall mariadb mariadb-devel

systemctl start mariadb systemctl enable mariadb

Then you can create the database:

% mysql -u root -p
MariaDB [(none)]> set password for 'root'@'localhost' = password('my password');
MariaDB [(none)]> create database redmine character set utf8;
MariaDB [(none)]> create user 'redmine'@'localhost' identified by 'somepass';
MariaDB [(none)]> grant all privileges on redmine.* to 'redmine'@'localhost';

Redmine configuration

Edit the configuration file from the sample:

% cd /var/www/redmine/config/
% cp database.yml.example database.yml

and edit to select the postgresql adapter:

production:
  adapter: postgresql
  database: redmine
  host: localhost
  username: redmine
  password: somepassword

Add the gem bundler:

% gem install bundler
Fetching: bundler-1.16.1.gem (100%)
Successfully installed bundler-1.16.1
Parsing documentation for bundler-1.16.1
Installing ri documentation for bundler-1.16.1
Done installing documentation for bundler after 5 seconds
1 gem installed

The documentation recommends to update the Gemfile, but that does not appear necessary on a modern version of Redmine. So on to installing the dependencies:

% cd /var/www/redmine
% bundle install

Note that if postgresql-devel is not installed, you get:

Installing pg 0.18.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory: /usr/local/lib/ruby/gems/2.5.0/gems/pg-0.18.4/ext

/usr/local/bin/ruby -r ./siteconf20180131-22563-n36glk.rb extconf.rb checking for pg_config... yes Using config values from /usr/bin/pg_config checking for libpq-fe.h... no Can't find the 'libpq-fe.h header *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options.

Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/usr/local/bin/$(RUBY_BASE_NAME) --with-pg --without-pg --enable-windows-cross --disable-windows-cross --with-pg-config --without-pg-config --with-pg_config --without-pg_config --with-pg-dir --without-pg-dir --with-pg-include --without-pg-include=${pg-dir}/include --with-pg-lib --without-pg-lib=${pg-dir}/lib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

/usr/local/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/pg-0.18.4/mkmf.log

extconf failed, exit code 1 Gem files will remain installed in /usr/local/lib/ruby/gems/2.5.0/gems/pg-0.18.4 for inspection. Results logged to /usr/local/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/pg-0.18.4/gem_make.out

An error occurred while installing pg (0.18.4), and Bundler cannot continue. Make sure that `gem install pg -v '0.18.4'` succeeds before bundling.

In Gemfile: pg

Interestingly, I learned about a nice feature of yum that I did not know about, which is that you can install based on a file, without knowing which package this is :

% yum install /usr/include/libpq-fe.h

So the alternative is to

% yum install postgresql-devel

After that, % bundle install should give you the following message:

Bundle complete! 32 Gemfile dependencies, 73 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Post-install message from yard:
--------------------------------------------------------------------------------
As of YARD v0.9.2:

RubyGems "--document=yri,yard" hooks are now supported. You can auto-configure YARD to automatically build the yri index for installed gems by typing:

$ yard config --gem-install-yri

See `yard config --help` for more information on RubyGems install hooks.

You can also add the following to your .gemspec to have YARD document your gem on install:

spec.metadata["yard.run"] = "yri" # use "yard" to build full HTML docs.

--------------------------------------------------------------------------------

Following up on the suggestion about yard:

% yard config --gem-install-yri
Updated ~/.gemrc: 'gem: --document=yri'

Then update /var/www/redmine/config/environment.rb with

ENV['RAILS_ENV'] ||= 'production'

Is that really needed?

The documentation then suggests using the deprecated command:

% RAILS_ENV=production bundle exec rake generate_session_store
Note: The rake task generate_session_store has been deprecated, please use the replacement version generate_secret_token

So let's do the "right" thing:

% RAILS_ENV=production bundle exec rake generate_secret_token

and then migrating the database:

% RAILS_ENV=production bundle exec rake db:migrate

This does not work without fixing permissions:

rake aborted!
PG::ConnectionBad: FATAL:  Ident authentication failed for user "postgres"
/var/www/redmine/app/models/custom_field.rb:38:in `'
/var/www/redmine/app/models/custom_field.rb:18:in `'
/var/www/redmine/lib/redmine/field_format.rb:108:in `field_attributes'
/var/www/redmine/lib/redmine/field_format.rb:111:in `'
/var/www/redmine/lib/redmine/field_format.rb:57:in `'
/var/www/redmine/lib/redmine/field_format.rb:21:in `'
/var/www/redmine/lib/redmine/field_format.rb:20:in `'
/var/www/redmine/lib/redmine.rb:40:in `'
/var/www/redmine/config/initializers/30-redmine.rb:6:in `'
/var/www/redmine/config/environment.rb:16:in `'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `
' Tasks: TOP => db:migrate => environment

I also tried as postgres user, but that does not work any better:

% su - postgres
Last login: Wed Jan 31 10:35:47 EST 2018 on pts/2
-bash-4.2$ cd /var/www/redmine/
-bash-4.2$ RAILS_ENV=production bundle exec rake db:migrate
Rails Error: Unable to access log file. Please ensure that /var/www/redmine/log/production.log exists and is writable (ie, make it writable for user and group: chmod 0664 /var/www/redmine/log/production.log). The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
rake aborted!
PG::ConnectionBad: FATAL:  Ident authentication failed for user "postgres"
/var/www/redmine/app/models/custom_field.rb:38:in `'
/var/www/redmine/app/models/custom_field.rb:18:in `'
/var/www/redmine/lib/redmine/field_format.rb:108:in `field_attributes'
/var/www/redmine/lib/redmine/field_format.rb:111:in `'
/var/www/redmine/lib/redmine/field_format.rb:57:in `'
/var/www/redmine/lib/redmine/field_format.rb:21:in `'
/var/www/redmine/lib/redmine/field_format.rb:20:in `'
/var/www/redmine/lib/redmine.rb:40:in `'
/var/www/redmine/config/initializers/30-redmine.rb:6:in `'
/var/www/redmine/config/environment.rb:16:in `'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `
' Tasks: TOP => db:migrate => environment (See full trace by running task with --trace)

Configuring a redmine user

Adding the user:

% useradd redmine
% chown -R redmine /var/www/redmine

That's not enough, still failure connecting to database.

You also need to make sure that user 'redmine' is the owner of the database:

postgres=# create database redmine with encoding 'UTF-8' owner redmine;
CREATE DATABASE
postgres=# alter user redmine superuser createrole createdb replication;
ALTER ROLE

I just love how the syntax is so order-dependent:

postgres=# create database redmine owner redmine;
CREATE DATABASE
create database redmine owner redmine with encoding 'UTF-8';
ERROR:  syntax error at or near "with encoding"
LINE 1: create database redmine owner redmine with encoding 'UTF-8';

There are suggestions on how to edit /var/lib/pgsql/data/pg_hba.conf, and modifying the localhost entry. That still does not work on RHEL 7.4. The /var/lib/pgsql/data/pg_log/postgresql-Wed.log shows

LOG:  could not connect to Ident server at address "::1", port 113: Connection refused

Turns out that the problem is that RHEL7.4 defaults localhost to IPv6, so need to add trust for the IPv6 localhost in /var/lib/pgsql/data/pg_hba.conf.

Now the following works:

% RAILS_ENV=production bundle exec rake db:migrate
== 1 Setup: migrating =========================================================
-- create_table("attachments", {:force=>true})
   -> 0.0586s
-- create_table("auth_sources", {:force=>true})
   -> 0.0333s
-- create_table("custom_fields", {:force=>true})
   -> 0.0666s
-- create_table("custom_fields_projects", {:id=>false, :force=>true})
   -> 0.0018s
-- create_table("custom_fields_trackers", {:id=>false, :force=>true})
   -> 0.0014s
-- create_table("custom_values", {:force=>true})
   -> 0.0966s
-- create_table("documents", {:force=>true})
   -> 0.0332s
-- add_index("documents", ["project_id"], {:name=>"documents_project_id"})
...

== 20170309214320 AddProjectDefaultAssignedToId: migrating ==================== -- add_column(:projects, :default_assigned_to_id, :integer, {:default=>nil}) -> 0.0008s -- column_exists?(:projects, :default_assignee_id, :integer) -> 0.0014s == 20170309214320 AddProjectDefaultAssignedToId: migrated (0.0024s) ===========

== 20170320051650 ChangeRepositoriesExtraInfoLimit: migrating ================= == 20170320051650 ChangeRepositoriesExtraInfoLimit: migrated (0.0000s) ========

== 20170418090031 AddViewNewsToAllExistingRoles: migrating ==================== == 20170418090031 AddViewNewsToAllExistingRoles: migrated (0.0080s) ===========

== 20170419144536 AddViewMessagesToAllExistingRoles: migrating ================ == 20170419144536 AddViewMessagesToAllExistingRoles: migrated (0.0069s) =======

Then the (probably optional) default data:

% RAILS_ENV=production bundle exec rake redmine:load_default_data

Select language: ar, az, bg, bs, ca, cs, da, de, el, en, en-GB, es, es-PA, et, eu, fa, fi, fr, gl, he, hr, hu, id, it, ja, ko, lt, lv, mk, mn, nl, no, pl, pt, pt-BR, ro, ru, sk, sl, sq, sr, sr-YU, sv, th, tr, uk, vi, zh, zh-TW [en] en ==================================== Default configuration data loaded.

Need to create the entry point for Redmine, copied /var/www/redmine/public/dispatch.fcgi.example as /var/www/redmine/public/dispatch.fcgi and then changed the ruby environment line to point to the one I had compiled:

#!/usr/local/bin/ruby

Make it executable:

chmod 755 public/dispatch.cgi

Create an Apache configuration file /etc/httpd/conf.d/redmine.conf with stuff lifted in part from the .htaccess provided with Redmine (remove the MaxRequestLen option :


    ServerName myservername
    ServerAdmin myadmin@mycompany.com
    DocumentRoot /var/www/redmine/public/
    ErrorLog logs/redmine_error_log

#If you are using mod_fcgid and are going to upload files larger than #131072 bytes you should consider adding the following line #that allows to upload files up to 20 mb Options Indexes ExecCGI FollowSymLinks Order allow,deny Allow from all AllowOverride all AddHandler fastcgi-script .fcgi AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI RewriteEngine On RewriteRule ^$ index.html [QSA] RewriteRule ^([^.]+)$ $1.html [QSA] RewriteCond %{REQUEST_FILENAME} !-f RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] RewriteRule ^(.*)$ dispatch.fcgi [QSA,L] ErrorDocument 500 "/500.html" ErrorDocument 404 "/404.html"

I still get the main RHEL page, and if I fetch the FCGI itself, I get the source code for the script, it does not get executed.

CGI script enablement

Checking if Apache has the fastcgi module:

% apachectl -M

No answer, option not listed in the man page. So

% apachectl fullstatus
The 'links' package is required for this functionality.

Installing links, then total lack of success

% apachectl fullstatus
                                 Page not found

The page you were trying to access doesn't exist or has been removed. [1]Back

References

Visible links 1. javascript:history.back()

OK. Approach number #2:

% httpd -M
Loaded Modules:
 core_module (static)
 so_module (static)
 http_module (static)
 access_compat_module (shared)
 ...

Is there a fast CGI module? Nope:

% httpd -M | grep cgi
 proxy_fcgi_module (shared)
 proxy_scgi_module (shared)
 cgi_module (shared)

According to instructions, need to add some dependencies for building:

% yum install libtool httpd-devel apr-devel apr

Then the build itself:

cd /usr/local/src/
wget http://www.fastcgi.com/dist/mod_fastcgi-current.tar.gz

tar -zxvf mod_fastcgi-current.tar.gz cd mod_fastcgi-2.4.6/ cp Makefile.AP2 Makefile make top_dir=/usr/lib/httpd make install top_dir=/usr/lib/httpd

Strike that. Better solution is:

% yum install mod_fcgid

Getting 400 errors (Bad Request). In the redmine_error_log, I see:

[Thu Feb 01 08:40:22.868890 2018] [authz_core:error] [pid 13062] [client 192.168.77.22:58420] AH01630: client denied by server configuration: /var/www/redmine/public/.html

Had to go to trace level 3 in Apache to find something:

auth phase 'translate' gave status 400: /dispatch.fcgi

What does that mean now? I find it amazing that something could end up in a "Bad Request" with nary a bleep in the Apache error log...

Ran the dispatch.fcgi locally:

% ./dispatch.fcgi
/usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:274:in `require': cannot load such file -- fcgi (LoadError)
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:274:in `block in require'
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:240:in `load_dependency'
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:274:in `require'
	from /usr/local/lib/ruby/gems/2.5.0/gems/rack-1.6.8/lib/rack/handler/fastcgi.rb:1:in `'
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:274:in `require'
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:274:in `block in require'
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:240:in `load_dependency'
	from /usr/local/lib/ruby/gems/2.5.0/gems/activesupport-4.2.8/lib/active_support/dependencies.rb:274:in `require'
	from ./dispatch.fcgi:20:in `
'

OK, finally found an interesting idea based on that message. It looks like I need to add {tt ""fcgi""} to the Gemfile and re-run bundle install. But that fails:

Fetching fcgi 0.9.2.1
Installing fcgi 0.9.2.1 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory: /usr/local/lib/ruby/gems/2.5.0/gems/fcgi-0.9.2.1/ext/fcgi

/usr/local/bin/ruby -r ./siteconf20180201-16089-15rs6mj.rb extconf.rb checking for fcgiapp.h... no checking for fastcgi/fcgiapp.h... no

current directory: /usr/local/lib/ruby/gems/2.5.0/gems/fcgi-0.9.2.1/ext/fcgi make "DESTDIR=" clean make: *** No rule to make target `clean'. Stop.

current directory: /usr/local/lib/ruby/gems/2.5.0/gems/fcgi-0.9.2.1/ext/fcgi make "DESTDIR=" make: *** No targets. Stop.

make failed, exit code 2

Gem files will remain installed in /usr/local/lib/ruby/gems/2.5.0/gems/fcgi-0.9.2.1 for inspection. Results logged to /usr/local/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/fcgi-0.9.2.1/gem_make.out

An error occurred while installing fcgi (0.9.2.1), and Bundler cannot continue. Make sure that `gem install fcgi -v '0.9.2.1'` succeeds before bundling.

In Gemfile: fcgi

Annoying. Another install:

% yum install fcgi-devel
No package fcgi-devel available.
Error: Nothing to do

OK. Trying to use the EPEL version then:

% cd /tmp
% wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
% rpm -ivh epel-release-latest-7.noarch.rpm

Then getting the fcgi-devel package succeeds:

% yum install fcgi-devel

Then backtracking a bit.

% bundle install

Now at least running ./dispatch.fcgi manually works. But the Bad Request is still there.

If I remove the following rewrite rules, then instead of a Bad Request, I get an Internal error

    RewriteRule ^$ index.html [QSA]
    RewriteRule ^([^.]+)$ $1.html [QSA]
    RewriteCond %{REQUEST_FILENAME} !-f
    
	RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]
    
    
	RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]
    

Does that mean the rewrite module has a bug?

OK. It turns out it's the .htaccess sample that has some error, there is a missing leading / in /dispatch.fcgi. After that, what I get is:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at rhspice-team@redhat.com to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.

If I write to a file from within the dispatch.fcgi file, it's clearly only called if I run it manually, not from the httpd server. Something is blocking it. Nothing in the error_log from Apache or Redmine, beyond:

[Thu Feb 01 12:20:32.575195 2018] [fcgid:info] [pid 19905] mod_fcgid: server rhel74-muse:/var/www/redmine/public/dispatch.fcgi(20265) started
[Thu Feb 01 12:20:32.576718 2018] [fcgid:info] [pid 19905] mod_fcgid: server rhel74-muse:/var/www/redmine/public/dispatch.fcgi(20266) started
[Thu Feb 01 12:20:38.582928 2018] [fcgid:info] [pid 19905] mod_fcgid: process /var/www/redmine/public/dispatch.fcgi(20266) exit(communication error), terminated by calling exit(), return code: 255
[Thu Feb 01 12:20:38.582987 2018] [fcgid:info] [pid 19905] mod_fcgid: process /var/www/redmine/public/dispatch.fcgi(20265) exit(communication error), terminated by calling exit(), return code: 255

When I run it manually, I get return code 0.

Maybe SELinux is getting in the way again.

% sestatus -b | grep httpd | grep cgi
httpd_enable_cgi                            on

So that seems to be allowed.

Looked at the docker image that is running Redmine to see what process is running. Apparently, the web server is NOT Apache:

% docker exec -it 9 bash
root@96ac71a4560b:/usr/src/redmine# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
redmine      1     0  0  2017 ?        00:02:45 /usr/local/bin/ruby bin/rails server -b 0.0.0.0
root      1546     0  0 17:39 ?        00:00:00 bash
root      1573  1546  0 17:45 ?        00:00:00 ps -ef

So apparently I also need to manually run the Rails server.

% /usr/local/bin/ruby bin/rails server -b 0.0.0.0 -e production
=> Booting WEBrick
=> Rails 4.2.8 application starting in production on http://0.0.0.0:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
[2018-02-01 12:46:34] INFO  WEBrick 1.4.2
[2018-02-01 12:46:34] INFO  ruby 2.5.0 (2017-12-25) [x86_64-linux]
...
/usr/local/lib/ruby/2.5.0/socket.rb:201:in `bind': Address already in use - bind(2) for 0.0.0.0:3000 (Errno::EADDRINUSE)

Removing the following from redmine.conf:

Listen *:3000

Then the server starts correctly:

 /usr/local/bin/ruby bin/rails server -b 0.0.0.0 -e production
=> Booting WEBrick
=> Rails 4.2.8 application starting in production on http://0.0.0.0:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
[2018-02-01 12:49:02] INFO  WEBrick 1.4.2
[2018-02-01 12:49:02] INFO  ruby 2.5.0 (2017-12-25) [x86_64-linux]
[2018-02-01 12:49:02] INFO  WEBrick::HTTPServer#start: pid=21470 port=3000

So maybe now the dispatcher works? Nope. But connecting to https://rhel74-muse:3000 now works! So I guess all that crap was for nothing.

Dumping the redmine database:

% pg_dump --host=localhost --username=redmine --dbname=redmine --file=redmine-db.sql

Restoring it:

% psql --file=/tmp/redmine/redmine-db.sql --host=localhost --username=redmine --dbname=redmine

Problem is that this runs into a number of issues:

SET
psql:/tmp/redmine/redmine-db.sql:9: ERROR:  unrecognized configuration parameter "lock_timeout"
psql:/tmp/redmine/redmine-db.sql:10: ERROR:  unrecognized configuration parameter "idle_in_transaction_session_timeout"
SET
SET
SET
SET
psql:/tmp/redmine/redmine-db.sql:15: ERROR:  unrecognized configuration parameter "row_security"
CREATE EXTENSION
COMMENT
SET
SET
SET
CREATE TABLE
ALTER TABLE
psql:/tmp/redmine/redmine-db.sql:70: ERROR:  syntax error at or near "AS"
LINE 2:     AS integer
            ^
psql:/tmp/redmine/redmine-db.sql:73: ERROR:  relation "attachments_id_seq" does not exist
psql:/tmp/redmine/redmine-db.sql:79: ERROR:  relation "attachments_id_seq" does not exist
CREATE TABLE
ALTER TABLE
psql:/tmp/redmine/redmine-db.sql:118: ERROR:  syntax error at or near "AS"
LINE 2:     AS integer
            ^
psql:/tmp/redmine/redmine-db.sql:121: ERROR:  relation "auth_sources_id_seq" does not exist
psql:/tmp/redmine/redmine-db.sql:127: ERROR:  relation "auth_sources_id_seq" does not exist
CREATE TABLE
ALTER TABLE
psql:/tmp/redmine/redmine-db.sql:159: ERROR:  syntax error at or near "AS"
LINE 2:     AS integer

Issues with the versions of Postgres, my docker image has:

% psql --version
psql (PostgreSQL) 10.0

The RHEL 7.4 VM has:

%  psql --version
psql (PostgreSQL) 9.2.23

So that means upgrading the database too. Apparently not very hard:

% yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-redhat10-10-1.noarch.rpm

Copying files from /usr/src/redmine/files.

The docker route

Self-explanatory:

% docker
bash: docker: command not found...
% curl -sSL https://get.docker.com/ | sh
  1. Executing docker install script, commit: 1d31602

WARNING: rhel is now only supported by Docker EE Check https://store.docker.com for information on Docker EE

So basically you have to pay for docker, I guess.

Back from DevConf.czDaily

Tuesday, January 30, 2018

Back from Brno, and full of positive energy

Mail

Horribly backed up on mail.

  • Replied to Lukas about coding style
  • Comment on memo-list about James. Ran into an issue with mu4e, made the mu4e-set-account function interactive to be able to workaround it manually. If that persists, I'll probably have to hook that function to some other mu4e function (reply-to path?)

Code reviews

Looked at a few patches from fziglio. Acked a couple easy ones.

Applied and tested the better way to reset the connexion. Unfortunately, at my first test, I had:

[626004.534889] qemu-system-x86[14681]: segfault at 4 ip 00005612cf06de58 sp 00007fffb41ec710 error 4 in qemu-system-x86_64[5612ced2d000+838000]

[626005.827507] pci-stub 0000:03:00.1: claimed by stub

Not much information in /var/log/libvirt/qemu/f25-turbo.log:

inputs_connect: inputs channel client create
red_qxl_set_cursor_peer:
2018-01-30 13:02:10.104+0000: shutting down, reason=crashed

In /var/log/libvirt/libvirt.log, all I have is:

2018-01-30 12:46:28.228+0000: 1196: error : qemuMonitorIO:697 : internal error: End of file from qemu monitor
2018-01-30 12:46:57.730+0000: 1215: warning : qemuDomainObjTaint:4168 : Domain id=9 name='f25-turbo' uuid=ede29304-fe0c-4ca4-abcd-d246481acd18 is tainted: custom-argv
2018-01-30 13:02:08.903+0000: 1196: error : qemuAgentIO:652 : internal error: End of file from agent monitor
2018-01-30 13:02:08.903+0000: 1196: error : qemuMonitorIO:697 : internal error: End of file from qemu monitor

Not sure if this is related to the patch. Trying again.

Tried again. The crash occured this time when I hit the CMD key on the keyboard.

Going back to master for the spice-streaming-agent, I then see the crash I had earlier, but no QEMU crash, the VM remains alive.

DISPLAY=:1 spice-streaming-agent -c noblock=yes

spice-streaming-agent[2240]: UNKNOWN msg of type 5 spice-streaming-agent[2240]: BAD VERSION 0 (expected is 1) spice-streaming-agent[2240]: BAD VERSION 108 (expected is 1) spice-streaming-agent[2240]: BAD VERSION 97 (expected is 1) spice-streaming-agent[2240]: read command from device FAILED -- read 1 expected 8 spice-streaming-agent[2240]: FAILED to read command

So the patch is not really making things better.

Done with the DevConf.cz SPICE talkDaily

Friday, January 19, 2018

Done with the DevConf.cz SPICE talk, except for fine-tuning of the timing.

As with each time I use Tao3D a bit extensively, I invent new techniques or fix things. Notably this time:

  • I figured out a way to do stepping initialization correctly

    * I added a nice looking virtual laser pointer with developer coordinates

I also ran into a couple of interesting issues:

  • Some code I added to detect if movie files were missing creates a large number of text objects. Running overnight, Tao3D crashed twice in the garbage collector for the text pool. This happened immediately after I upated to the latest OSX, had not seen that before. I mention the update because the crash is in the dtor for std::string, and it's possible something subtly changed there that exposed an issue I had before.

SPICE demo

Trying to setup the demo using Snir's VM. Running into some issues with the SPICE client and mouse control. I only had that after rebooting the VM due to a hang in the spice streaming agent after killing it (sometimes happened to me here too).

On turbo, I still have:

spice-streaming-agent[886]: write_all -- 1939 bytes written
spice-streaming-agent[886]: wrote data msg body of size 1939
spice-streaming-agent[886]: BAD VERSION 97 (expected is 1)
spice-streaming-agent[886]: got a frame -- size is 1939 (31 ms) (33 ms from last frame)(1362 us)
spice-streaming-agent[886]: write_all -- 8 bytes written
spice-streaming-agent[886]: wrote 8 bytes of header of data msg with frame of size 1939 bytes
spice-streaming-agent[886]: write_all -- 1939 bytes written
spice-streaming-agent[886]: wrote data msg body of size 1939
spice-streaming-agent[886]: read command from device FAILED -- read 1 expected 8
spice-streaming-agent[886]: FAILED to read command

quite rapidly after starting. Frediano suggested I try the

Yum vs dnf

I took some flak when I dared writing on my blog that dnf was one too many package managers. I stand by my opinion. Now, each time I revert to RHEL, I end up

Love my 4K monitor

The 4K monitor I received proves incredibly helpful. Four times as much space to layout windows. I used to have a full HD in portrait mode to have more code lines on the screen. No longer. Now, I have plenty lines in my Emacs buffers, but I can have twice as many windows open on the screen. Maybe too many...

Keep goingDaily

Friday, January 12, 2018

Building MediaSDKDaily

Thursday, January 11, 2018

Have to build without "warnings as error" using:

% perl tools/builder/build_mfx.pl --cmake=intel64.make.release --no-warn-as-error

Builds fails with:

../../__lib/release/libencode_hw.a(mfx_h264_scd.cpp.o): In function `MfxHwH264Encode::ASC::Get_CpuFeature_AVX2()':
/home/ddd/Work/msdk/_studio/mfx_lib/encode_hw/h264/src/mfx_h264_scd.cpp:582: undefined reference to `__cpu_model'
../../__lib/release/libencode_hw.a(mfx_h264_scd.cpp.o): In function `MfxHwH264Encode::ASC::Get_CpuFeature_SSE41()':
mfx_h264_scd.cpp:(.text+0x4f43): undefined reference to `__cpu_model'
../../__lib/release/libencode_hw.a(mfx_h264_scd.cpp.o): In function `_GLOBAL__sub_I_mfx_h264_scd.cpp':
mfx_h264_scd.cpp:(.text.startup+0x2d): undefined reference to `__cpu_model'
../../__lib/release/libvpp_hw.a(mfx_scd.cpp.o): In function `SceneChangeDetector::Init(int, int, int, unsigned int, CmDevice*)':
/home/ddd/Work/msdk/_studio/shared/src/mfx_scd.cpp:12008: undefined reference to `__cpu_model'

Fix is adding -lgcc at end of build line:

% cd ~/Work/msdk/__cmake/intel64.make.release/_studio/mfx_lib
% /usr/bin/c++ -fPIC -msse4.2 -std=c++11 -m64 -g -pipe -fPIC -O2 -D_FORTIFY_SOURCE=2 -fstack-protector -Wall -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-unused  -Wformat -Wformat-security -DNDEBUG -L/usr/local/lib -Wl,--no-undefined,-z,relro,-z,now,-z,noexecstack -Wl,--default-symver -Wl,--version-script=/home/ddd/Work/msdk/_studio/mfx_lib/libmfx.map -fstack-protector   -Wl,--no-undefined,-z,relro,-z,now,-z,noexecstack -fstack-protector  -shared -Wl,-soname,libmfxhw64.so -o ../../__bin/release/libmfxhw64.so CMakeFiles/mfxhw64.dir/scheduler/src/mfx_scheduler_core.cpp.o CMakeFiles/mfxhw64.dir/scheduler/src/mfx_scheduler_core_ischeduler.cpp.o CMakeFiles/mfxhw64.dir/scheduler/src/mfx_scheduler_core_iunknown.cpp.o CMakeFiles/mfxhw64.dir/scheduler/src/mfx_scheduler_core_task.cpp.o CMakeFiles/mfxhw64.dir/scheduler/src/mfx_scheduler_core_task_management.cpp.o CMakeFiles/mfxhw64.dir/scheduler/src/mfx_scheduler_core_thread.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_async.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_decode.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_enc.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_encode.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_pak.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_plugin.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_query.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_session.cpp.o CMakeFiles/mfxhw64.dir/shared/src/libmfxsw_vpp.cpp.o CMakeFiles/mfxhw64.dir/shared/src/mfx_session.cpp.o CMakeFiles/mfxhw64.dir/shared/src/mfx_user_plugin.cpp.o CMakeFiles/mfxhw64.dir/shared/src/mfx_brc_common.cpp.o CMakeFiles/mfxhw64.dir/shared/src/mfx_mpeg2_dec_common.cpp.o CMakeFiles/mfxhw64.dir/shared/src/mfx_vc1_dec_common.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/cm_mem_copy.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/mfx_vpp_vaapi.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/libmfx_allocator.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/libmfx_allocator_vaapi.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/libmfx_core.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/libmfx_core_factory.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/libmfx_core_vaapi.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/mfx_umc_alloc_wrapper.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/mfx_static_assert_structs.cpp.o CMakeFiles/mfxhw64.dir/__/shared/src/mfx_mfe_adapter.cpp.o -Xlinker --start-group ../../__lib/release/libmfx_trace_hw.a ../../__lib/release/libbitrate_control.a ../../__lib/release/libh264_dec_hw.a ../../__lib/release/libjpeg_common.a ../../__lib/release/libjpeg_dec_hw.a ../../__lib/release/libmpeg2_dec_hw.a ../../__lib/release/libumc.a ../../__lib/release/libvm.a ../../__lib/release/libvm_plus.a ../../__lib/release/libumc_va_hw.a ../../__lib/release/libcmrt_cross_platform_hw.a ../../__lib/release/libdecode_hw.a ../../__lib/release/libencode_hw.a ../../__lib/release/libh264_la.a ../../__lib/release/libgenx_h264_encode_embeded.a ../../__lib/release/libmfx_common.a ../../__lib/release/libmfx_common_hw.a ../../__lib/release/libvpp_hw.a ../../__lib/release/libh264_common.a ../../__lib/release/libh264_preenc.a ../../__lib/release/libh264_enc.a ../../__lib/release/libh264_pak.a ../../__lib/release/libvc1_common.a ../../__lib/release/libvc1_dec_hw.a ../../__lib/release/libmfx_trace.a -lpthread -ldl -lva ../../__lib/release/libSafeString.a -Xlinker --end-group -lgcc

4K monitor still not arrived. Sigh. Checked the Amazon tracking, they say they "attempted to deliver". But no delivery slip, and I was there the whole day, and specifically at the announced time (11:58). Grmbl.

Redmine reinstallDaily

Tuesday, January 9, 2018

yum -y install postgresql postgresql-devel postgresql-server ImageMagick ImageMagick-devel libcurl-devel

httpd-devel

More continuous integrationDaily

Friday, December 15, 2017

I keep chugging on continuous integration. This time I would like the SPICE project to work, although I'm also looking at ELFE and XL.

Building SPICE with c3d/build

The first build from Jenkins was successful, which was honestly quite surprising, because I had put some extra operating systems and configurations which I did not expect to work OK.

The reason it worked was really that I had copied the project from recorder, and I had updated the Git project page, but that's not enough. The git repository was still recorder.git.