Valid certificate reported as expired

[Andrej Podzimek]

Hello,

both psql and pgAdmin refuse to connect to my PostgreSQL server using SSL. These two error messages alternate:

SSL error: sslv3 alert certificate expired
SSL error: certificate verify failed

CA certificate is valid till 2011.
Server certificate is valid till 2009.
Client certificate is valid till 2009.

So the first error message is obviously a nonsense.

I asked on the pgAdmin and PostgreSQL mailing lists. The answer was just about the same in both cases: This must be an OpenSSL issue.

In fact, the whole story is a bit more complicated:

1) I enabled OpenSSL for psql and pgAdmin in June 2008. It worked.
2) It stopped working (for the first time) at the end of August, with the certificate expired message.
3) Adding the CA certificate and CRL on the *client* side fixed this, amazingly.
4) Then it worked for about one month, till the beginning of October.
5) Stopped working again about two days ago, this time with two error messages.

Certificate and key files are still in place and computer clocks show correct time.

I have the 0.9.8i version installed. Should I try the h version again? (I am not sure whether the upgrade from h to i is related to the malfunction or not.)

Other programs, such as Courier-MTA, work just fine.

Is it possible to get more log messages? There is something wrong with the OpenSSL + PostgreSQL combination. There are two scenarios corresponding to the error messages mentioned above.
1) Server says the certificate has expired. Client says certificate verification failed.
2) Server says the client did not supply a certificate. Client says the certificate has expired.

Nobody says *which* certificate expired. (AFAIK, all of them are valid. Checked that twice.)

What could be wrong? Thank you in advance for any piece of advice.

Andrej
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Dr. Stephen Henson]

Post by Andrej Podzimek
Hello,
both psql and pgAdmin refuse to connect to my PostgreSQL server using SSL.
SSL error: sslv3 alert certificate expired
SSL error: certificate verify failed
CA certificate is valid till 2011.
Server certificate is valid till 2009.
Client certificate is valid till 2009.
So the first error message is obviously a nonsense.
I asked on the pgAdmin and PostgreSQL mailing lists. The answer was just
about the same in both cases: This must be an OpenSSL issue.
1) I enabled OpenSSL for psql and pgAdmin in June 2008. It worked.
2) It stopped working (for the first time) at the end of August, with the
certificate expired message.
3) Adding the CA certificate and CRL on the *client* side fixed this, amazingly.
4) Then it worked for about one month, till the beginning of October.
5) Stopped working again about two days ago, this time with two error messages.
Certificate and key files are still in place and computer clocks show correct time.
I have the 0.9.8i version installed. Should I try the h version again? (I
am not sure whether the upgrade from h to i is related to the malfunction
or not.)
Other programs, such as Courier-MTA, work just fine.
Is it possible to get more log messages? There is something wrong with the
OpenSSL + PostgreSQL combination. There are two scenarios corresponding to
the error messages mentioned above.
1) Server says the certificate has expired. Client says certificate verification failed.
2) Server says the client did not supply a certificate. Client says the
certificate has expired.
Nobody says *which* certificate expired. (AFAIK, all of them are valid.
Checked that twice.)
What could be wrong? Thank you in advance for any piece of advice.

Are any intermediate CA certificates involved?

This command will dump all certificates received:

openssl s_client -connect hostname:portnum -showcerts

If you split them into files and try:

openssl x509 -in cert.pem -dates -noout

It will print their dates.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Andrej Podzimek]

Post by Dr. Stephen Henson
Are any intermediate CA certificates involved?

No. The CA is home-made, created using OpenSSL. It has a self-signed certificate.

Post by Dr. Stephen Henson
openssl s_client -connect hostname:portnum -showcerts

[***@dustbin ~]$ openssl s_client -connect my.server.address:5432 -showcerts
CONNECTED(00000003)

Post by Dr. Stephen Henson
openssl x509 -in cert.pem -dates -noout

Can't try this right now...

As for the dates, I store all my certificates with the human-readable preamble (the -text option used with x509). Dates are OK in all of them. I dumped them again and got the same result.

This is what I tried next:

[***@dustbin ~]$ openssl s_client -debug -connect my.server.address:5432 -showcerts
CONNECTED(00000003)
write to 0x9fcb948 [0x9fcb990] (124 bytes => 124 (0x7C))
0000 - 80 7a 01 03 01 00 51 00-00 00 20 00 00 39 00 00 .z....Q... ..9..
0010 - 38 00 00 35 00 00 16 00-00 13 00 00 0a 07 00 c0 8..5............
0020 - 00 00 33 00 00 32 00 00-2f 00 00 07 05 00 80 03 ..3..2../.......
0030 - 00 80 00 00 05 00 00 04-01 00 80 00 00 15 00 00 ................
0040 - 12 00 00 09 06 00 40 00-00 14 00 00 11 00 00 08 ***@.........
0050 - 00 00 06 04 00 80 00 00-03 02 00 80 64 70 9c 33 ............dp.3
0060 - 54 71 07 96 37 d8 e5 9c-22 01 5b 19 60 9f d0 1f Tq..7...".[.`...
0070 - a3 43 82 8d 51 2d eb bc-c8 84 1c bb .C..Q-......
read from 0x9fcb948 [0x9fd0ef0] (7 bytes => 0 (0x0))
4407:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:188:

A local IP connection directly on the server fails the same way, too. (Non-SSL IP connections to the database do work, however.)

What should I try now? If you want me to carry out further experiments, just let me know.

Andrej
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Kyle Hamilton]

How about posting the certificate chain printed by -showcerts? If you
don't get one, then it's entirely possible that you've got a problem
on your server (such as not having the correct private key for the
certificate).

-Kyle H

Post by Andrej Podzimek

Post by Dr. Stephen Henson
Are any intermediate CA certificates involved?

No. The CA is home-made, created using OpenSSL. It has a self-signed certificate.

Post by Dr. Stephen Henson
openssl s_client -connect hostname:portnum -showcerts

CONNECTED(00000003)

Post by Dr. Stephen Henson
openssl x509 -in cert.pem -dates -noout

Can't try this right now...
As for the dates, I store all my certificates with the human-readable
preamble (the -text option used with x509). Dates are OK in all of them. I
dumped them again and got the same result.
my.server.address:5432 -showcerts
CONNECTED(00000003)
write to 0x9fcb948 [0x9fcb990] (124 bytes => 124 (0x7C))
0000 - 80 7a 01 03 01 00 51 00-00 00 20 00 00 39 00 00 .z....Q... ..9..
0010 - 38 00 00 35 00 00 16 00-00 13 00 00 0a 07 00 c0
8..5............
0020 - 00 00 33 00 00 32 00 00-2f 00 00 07 05 00 80 03
..3..2../.......
0030 - 00 80 00 00 05 00 00 04-01 00 80 00 00 15 00 00
................
0040 - 12 00 00 09 06 00 40 00-00 14 00 00 11 00 00 08
0050 - 00 00 06 04 00 80 00 00-03 02 00 80 64 70 9c 33
............dp.3
0060 - 54 71 07 96 37 d8 e5 9c-22 01 5b 19 60 9f d0 1f
Tq..7...".[.`...
0070 - a3 43 82 8d 51 2d eb bc-c8 84 1c bb .C..Q-......
read from 0x9fcb948 [0x9fd0ef0] (7 bytes => 0 (0x0))
A local IP connection directly on the server fails the same way, too.
(Non-SSL IP connections to the database do work, however.)
What should I try now? If you want me to carry out further experiments, just let me know.
Andrej
______________________________________________________________________
OpenSSL Project http://www.openssl.org

______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Andrej Podzimek]

Post by Kyle Hamilton
How about posting the certificate chain printed by -showcerts? If you
don't get one, then it's entirely possible that you've got a problem
on your server (such as not having the correct private key for the
certificate).

Well, that is possible, but not very probable. I am the only admin of the server. Keys and certificates on the server haven't been changed since July. SSL had worked with the same keys and certificates for a few months before this weird issue appeared.

Andrej
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Dr. Stephen Henson]

Post by Andrej Podzimek

Post by Dr. Stephen Henson
Are any intermediate CA certificates involved?

No. The CA is home-made, created using OpenSSL. It has a self-signed certificate.

Post by Dr. Stephen Henson
openssl s_client -connect hostname:portnum -showcerts

CONNECTED(00000003)
4386:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake

Hmmm.... Is that the right port for SSL/TLS if it is it looks like it isn't
just a a case of connecting to the right port to get an SSL/TLS connection.
Might be some STARTTLS equivalent but I'm not sure what it is for that
application.

Post by Andrej Podzimek
As for the dates, I store all my certificates with the human-readable
preamble (the -text option used with x509). Dates are OK in all of them. I
dumped them again and got the same result.

It looks like an expired certificate is somehow being used. How isn't clear at
this stage. If you have CA certificates in directories or files make sure an
old one isn't in there.

The best I can suggest at this point is modifying OpenSSL or the application to
dump out any expired certificates to a temp file so you can see which one(s)
it is complaining about.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Andrej Podzimek]

Post by Dr. Stephen Henson
Hmmm.... Is that the right port for SSL/TLS if it is it looks like it isn't
just a a case of connecting to the right port to get an SSL/TLS connection.
Might be some STARTTLS equivalent but I'm not sure what it is for that
application.

PostgreSQL always listens on one port. This is the only port I ever used for TCP/IP connections. So there must be something like STARTTLS, as it can handle both encrypted and unencrypted connections.

[***@charon ~]# netstat -atpn | grep postgres | grep LISTEN
tcp 0 0 10.84.53.1:5432 0.0.0.0:* LISTEN 9808/postgres
tcp 0 0 10.84.55.1:5432 0.0.0.0:* LISTEN 9808/postgres
tcp 0 0 217.112.173.73:5432 0.0.0.0:* LISTEN 9808/postgres
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 9808/postgres
tcp 0 0 2002:d970:ad49:2:::5432 :::* LISTEN 9808/postgres
tcp 0 0 2002:d970:ad49:1:::5432 :::* LISTEN 9808/postgres
tcp 0 0 2002:d970:ad49::1:5432 :::* LISTEN 9808/postgres
tcp 0 0 ::1:5432 :::* LISTEN 9808/postgres

Post by Dr. Stephen Henson
It looks like an expired certificate is somehow being used. How isn't clear at
this stage. If you have CA certificates in directories or files make sure an
old one isn't in there.

I only have one directory and one CA certificate. That makes the task simple.

On the client:
[***@xandrej .postgresql]$ openssl x509 -in postgresql.crt -text | grep Not
Not Before: Mar 25 12:00:00 2008 GMT
Not After : Mar 25 12:00:00 2009 GMT
[***@xandrej .postgresql]$ openssl x509 -in root.crt -text | grep Not
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT
On the server:
[***@charon data]# openssl x509 -in server.crt -text | grep Not
Not Before: Jul 23 09:20:00 2008 GMT
Not After : Jul 23 09:20:00 2009 GMT
[***@charon data]# openssl x509 -in root.crt -text | grep Not
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT

Post by Dr. Stephen Henson
The best I can suggest at this point is modifying OpenSSL or the application to
dump out any expired certificates to a temp file so you can see which one(s)
it is complaining about.

That would be helpful. But how could I do that? What file should I change? Is there a patch/howto?

Two more remarks:
1) Downgraded to h and restarted PostgreSQL today. (Grrr...) Still the same error.
2) Just a wild guess, a shot in the dark: Could this be a locale-related issue? Does OpenSSL use/parse text representations of dates and times? If so, getting (for example) '9. říj 15.12' instead of 'Oct 9 15:12' could result in a comparison failure if not handled properly. (But this is probably not the case. Presumably, a binary representation (such as epoch) is used...)

Andrej

______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Victor Duchovni]

Post by Andrej Podzimek
I only have one directory and one CA certificate. That makes the task simple.
Not Before: Mar 25 12:00:00 2008 GMT
Not After : Mar 25 12:00:00 2009 GMT
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT
Not Before: Jul 23 09:20:00 2008 GMT
Not After : Jul 23 09:20:00 2009 GMT
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT

When a PEM file holds multiple certificates (a chain), this command
only shows the first one. You need to break each of the ".crt" files
into separate files for each certificate, and look at those.

[Andrej Podzimek]

Post by Victor Duchovni

Post by Andrej Podzimek
I only have one directory and one CA certificate. That makes the task simple.
Not Before: Mar 25 12:00:00 2008 GMT
Not After : Mar 25 12:00:00 2009 GMT
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT
Not Before: Jul 23 09:20:00 2008 GMT
Not After : Jul 23 09:20:00 2009 GMT
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT

When a PEM file holds multiple certificates (a chain), this command
only shows the first one. You need to break each of the ".crt" files
into separate files for each certificate, and look at those.

The root.crt file holds exactly one self-signed CA certificate. This CA was then used to create postgresql.crt and server.crt. Each file contains exactly one certificate. There are no chains.

There is only one block like this in each file:
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

Should I try to append the CA certificate to the server and client certificate files? Some apps require this, but PostgreSQL worked just fine without it till the beginning of October.

Andrej
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Victor Duchovni]

Post by Andrej Podzimek

Post by Victor Duchovni
When a PEM file holds multiple certificates (a chain), this command
only shows the first one. You need to break each of the ".crt" files
into separate files for each certificate, and look at those.

The root.crt file holds exactly one self-signed CA certificate. This CA was
then used to create postgresql.crt and server.crt. Each file contains
exactly one certificate. There are no chains.
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Should I try to append the CA certificate to the server and client
certificate files? Some apps require this, but PostgreSQL worked just fine
without it till the beginning of October.

If the client and server's certificate files are fresh, the only other
certs that can be stale are the client or server's copies of the root
CA cert in CAfile or CApath.

Running "ssldump" or "wireshark" on a capture of the session will reveal
which certs are exchanged on the wire, and which side initiates the alert,
but it will not reveal which side has the stale root CA cert.

Do check your CAfile and CApath settings on both sides, ...

[Dr. Stephen Henson]

Post by Andrej Podzimek

Post by Dr. Stephen Henson
Hmmm.... Is that the right port for SSL/TLS if it is it looks like it isn't
just a a case of connecting to the right port to get an SSL/TLS
connection. Might be some STARTTLS equivalent but I'm not sure what it is
for that
application.

PostgreSQL always listens on one port. This is the only port I ever used
for TCP/IP connections. So there must be something like STARTTLS, as it can
handle both encrypted and unencrypted connections.
tcp 0 0 10.84.53.1:5432 0.0.0.0:* LISTEN
9808/postgres
tcp 0 0 10.84.55.1:5432 0.0.0.0:* LISTEN
9808/postgres
tcp 0 0 217.112.173.73:5432 0.0.0.0:* LISTEN
9808/postgres
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN
9808/postgres
tcp 0 0 2002:d970:ad49:2:::5432 :::* LISTEN
9808/postgres
tcp 0 0 2002:d970:ad49:1:::5432 :::* LISTEN
9808/postgres
tcp 0 0 2002:d970:ad49::1:5432 :::* LISTEN
9808/postgres
tcp 0 0 ::1:5432 :::* LISTEN
9808/postgres

Post by Dr. Stephen Henson
It looks like an expired certificate is somehow being used. How isn't clear at
this stage. If you have CA certificates in directories or files make sure an
old one isn't in there.

I only have one directory and one CA certificate. That makes the task simple.
Not Before: Mar 25 12:00:00 2008 GMT
Not After : Mar 25 12:00:00 2009 GMT
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT
Not Before: Jul 23 09:20:00 2008 GMT
Not After : Jul 23 09:20:00 2009 GMT
Not Before: Nov 12 16:03:00 2006 GMT
Not After : Nov 11 16:03:00 2011 GMT

Then I suggest you run the following command on those systems too:

openssl verify -CAfile root.crt other.crt

Where "other.crt" is the EE certificate, server.crt or posgresql.crt

Post by Andrej Podzimek

Post by Dr. Stephen Henson
The best I can suggest at this point is modifying OpenSSL or the application to
dump out any expired certificates to a temp file so you can see which one(s)
it is complaining about.

That would be helpful. But how could I do that? What file should I change?
Is there a patch/howto?

In crypto/x509/x509_vfy.c the function check_cert_time() is the one you need.
Around the line with X509_V_ERR_CERT_HAS_EXPIRED is the certificate it thinks
has expired "x". Suggest you dump that out to a temp file using
PEM_write_X509()

Post by Andrej Podzimek
1) Downgraded to h and restarted PostgreSQL today. (Grrr...) Still the same error.
2) Just a wild guess, a shot in the dark: Could this be a locale-related
issue? Does OpenSSL use/parse text representations of dates and times? If
so, getting (for example) '9. ??íj 15.12' instead of 'Oct 9 15:12' could
result in a comparison failure if not handled properly. (But this is
probably not the case. Presumably, a binary representation (such as epoch)
is used...)

OpenSSL just uses time() and gmtime_r() (or equivalent) and relies on the
contents of struct tm.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Andrej Podzimek]

Post by Dr. Stephen Henson
openssl verify -CAfile root.crt other.crt
Where "other.crt" is the EE certificate, server.crt or posgresql.crt

Says OK on both machines.

Post by Dr. Stephen Henson
In crypto/x509/x509_vfy.c the function check_cert_time() is the one you need.
Around the line with X509_V_ERR_CERT_HAS_EXPIRED is the certificate it thinks
has expired "x". Suggest you dump that out to a temp file using
PEM_write_X509()

Tried that. Added

#include<openssl/pem.h>

and modified the appropriate part of check_cert_time() as follows:

if (i < 0)
{
+ FILE * f;
+ f = fopen( "/tmp/CERTDUMP_EXPIRED", "w" );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_HAS_EXPIRED;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}

The four lines I added did *not* execute at all on errors. I tried that multiple times and restarted PostgreSQL to make sure libraries get reloaded. Wrote a dummy program that could really open the file for writing. OpenSSL did not even touch the file. Checked twice, compiled twice...

I even tried to recompile PostgreSQL (!) to make sure there is no static linking and the like. Nothing of that kind. It still didn't work. So I modified the whole function like this:

static int check_cert_time(X509_STORE_CTX *ctx, X509 *x)
{
time_t *ptime;
int i;
+ FILE * f;
+ f = fopen( "/tmp/CERTDUMP_EXPIRED", "w" );

if (ctx->param->flags & X509_V_FLAG_USE_CHECK_TIME)
ptime = &ctx->param->check_time;
else
ptime = NULL;

+ fputs( "Before comparison.", f );
i=X509_cmp_time(X509_get_notBefore(x), ptime);
if (i == 0)
{
+ fputs( "BEFORE FIELD ERROR", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_ERROR_IN_CERT_NOT_BEFORE_FIELD;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}

if (i > 0)
{
+ fputs( "NOT_YET failure", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_NOT_YET_VALID;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}

i=X509_cmp_time(X509_get_notAfter(x), ptime);
if (i == 0)
{
+ fputs( "AFTER FIELD ERROR", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_ERROR_IN_CERT_NOT_AFTER_FIELD;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}

if (i < 0)
{
+ fputs( "EXPIRED failure", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_HAS_EXPIRED;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}

return 1;
}

The result was surprising: The file /tmp/CERTDUMP_EXPIRED contained *only* 'Before comparison.'. This means that *none* of the further branches could run! (In such case, even fclose() did not run, but 'Before comparison.' was probably flushed automatically when the process exited.)

So it seems that timestamp evaluation is OK. The function probably reached its end and returned 1. Bud where does the error message come from?

Is there anything I am doing wrong? There are thousands of PostgreSQL users. Most of them probably need SSL. But there are no similar reports, AFAIK. :-(

Andrej
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Dr. Stephen Henson]

Post by Andrej Podzimek

Post by Dr. Stephen Henson
openssl verify -CAfile root.crt other.crt
Where "other.crt" is the EE certificate, server.crt or posgresql.crt

Says OK on both machines.

Post by Dr. Stephen Henson
In crypto/x509/x509_vfy.c the function check_cert_time() is the one you need.
Around the line with X509_V_ERR_CERT_HAS_EXPIRED is the certificate it thinks
has expired "x". Suggest you dump that out to a temp file using
PEM_write_X509()

Tried that. Added
#include<openssl/pem.h>
if (i < 0)
{
+ FILE * f;
+ f = fopen( "/tmp/CERTDUMP_EXPIRED", "w" );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_HAS_EXPIRED;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}
The four lines I added did *not* execute at all on errors. I tried that
multiple times and restarted PostgreSQL to make sure libraries get
reloaded. Wrote a dummy program that could really open the file for
writing. OpenSSL did not even touch the file. Checked twice, compiled
twice...
I even tried to recompile PostgreSQL (!) to make sure there is no static
linking and the like. Nothing of that kind. It still didn't work. So I
static int check_cert_time(X509_STORE_CTX *ctx, X509 *x)
{
time_t *ptime;
int i;
+ FILE * f;
+ f = fopen( "/tmp/CERTDUMP_EXPIRED", "w" );
if (ctx->param->flags & X509_V_FLAG_USE_CHECK_TIME)
ptime = &ctx->param->check_time;
else
ptime = NULL;
+ fputs( "Before comparison.", f );
i=X509_cmp_time(X509_get_notBefore(x), ptime);
if (i == 0)
{
+ fputs( "BEFORE FIELD ERROR", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_ERROR_IN_CERT_NOT_BEFORE_FIELD;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}
if (i > 0)
{
+ fputs( "NOT_YET failure", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_NOT_YET_VALID;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}
i=X509_cmp_time(X509_get_notAfter(x), ptime);
if (i == 0)
{
+ fputs( "AFTER FIELD ERROR", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_ERROR_IN_CERT_NOT_AFTER_FIELD;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}
if (i < 0)
{
+ fputs( "EXPIRED failure", f );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_HAS_EXPIRED;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}
return 1;
}
The result was surprising: The file /tmp/CERTDUMP_EXPIRED contained *only*
'Before comparison.'. This means that *none* of the further branches could
run! (In such case, even fclose() did not run, but 'Before comparison.' was
probably flushed automatically when the process exited.)
So it seems that timestamp evaluation is OK. The function probably reached
its end and returned 1. Bud where does the error message come from?
Is there anything I am doing wrong? There are thousands of PostgreSQL
users. Most of them probably need SSL. But there are no similar reports,
AFAIK. :-(

Have you enabled CRL checking too? You can also get that if the nextUpdate
time in a CRL has passed. That might explain things if the CRL runs for a
month or so.

That error is produced in s3_both.c, see the SSL_AD_CERTIFICATE_EXPIRED stuff.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Andrej Podzimek]

Post by Dr. Stephen Henson
Have you enabled CRL checking too? You can also get that if the nextUpdate
time in a CRL has passed. That might explain things if the CRL runs for a
month or so.

WOW! That's it! Thank you so much!

CRL expired exactly the day it stopped working. I did not know that a CRL could expire. Never explored it with openssl crl -text ... before. Now I know it has those two dates, just like a certificate. Added CRL updates to cron tables on the server and all the clients.

I would like to apologize for all the mess around this issue. This was simply my mistake. (However, it would be nice if the error message said 'CRL expired' instead of 'certificate expired'. Had I seen 'CRL' in the message, I would have checked that first.)

Once more many thanks for your advice. My db connection works again.

Andrej

______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org

[Victor Duchovni]

Post by Andrej Podzimek
if (i < 0)
{
+ FILE * f;
+ f = fopen( "/tmp/CERTDUMP_EXPIRED", "w" );
+ PEM_write_X509( f, x );
+ fclose( f );
ctx->error=X509_V_ERR_CERT_HAS_EXPIRED;
ctx->current_cert=x;
if (!ctx->verify_cb(0, ctx))
return 0;
}

Don't open the file with mode "w", open it for append. Multiple
certs are checked (root and server or client) and you don't want
results from later checks to clobber the contents of the file.

[Dave Thompson]

-----Original Message-----
Sent: Thursday, 09 October, 2008 10:39

Post by Dr. Stephen Henson
Might be some STARTTLS equivalent but I'm not sure what it is for that
application.

PostgreSQL always listens on one port. This is the only port I
ever used for TCP/IP connections. So there must be something like
STARTTLS, as it can handle both encrypted and unencrypted connections.

That would explain why openssl s_client got handshake failure. STARTTLS
logic is specific per app protocol and s_client doesn't know postgresql.

Post by Dr. Stephen Henson
The best I can suggest at this point is modifying OpenSSL or

the application to

Post by Dr. Stephen Henson
dump out any expired certificates to a temp file so you can see

which one(s)

Post by Dr. Stephen Henson
it is complaining about.

That would be helpful. But how could I do that? What file should
I change? Is there a patch/howto?

Can you just run (commandline) openssl s_server on the server,
listening on some port of your choice, with at least -showcerts,
and openssl s_client on the client to talk to that port, ditto?
And for each specifying the right cert/keyfiles, and for s_server
-verify 1 since you indicate the real server requires client auth.
And if the real programs use a specific protocol (2/3/tls) specify
that. This should exercise the same openssl protocol logic as
the real client to the real server, but with a nice display.
Preferably the commandline on each system should be the same
version as the library used by/in the app on that system.

1) Downgraded to h and restarted PostgreSQL today.
(Grrr...) Still the same error.
2) Just a wild guess, a shot in the dark: Could this be a
locale-related issue? Does OpenSSL use/parse text representations
of dates and times? If so, getting (for example) '9. říj 15.12'
instead of 'Oct 9 15:12' could result in a comparison failure if
not handled properly. (But this is probably not the case.
Presumably, a binary representation (such as epoch) is used...)

The representation in the certificate is the ASN.1 DER encoding,
which is all digits except for a +, -, or letter Z for timezone.
I wouldn't call it binary exactly -- it's NOT a C-lib time_t,
for example -- but it is locale-independent.



______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users-MCmKBN63+***@public.gmane.org
Automated List Manager majordomo-MCmKBN63+***@public.gmane.org